Advice Request AV DB Engine

[Mr.NoName]

What database engine AV companies uses for their malware database. "sqlite, Access, Excell , csv, realm" or something else ?

 

[Mr.NoName]

And no response..... What a community.

 

[Kongo]

And no response..... What a community.

 

[Guilhermesene]

What database engine AV companies uses for their malware database. "sqlite, Access, Excell , csv, realm" or something else ?

In my view, I think it is difficult to know the exact answer to this, more likely a Mysql, PostgreSQL or some database that focuses on Security without hindering performance.

 

[Shadowra]

And no response..... What a community.

Yes, the community is good
Either people haven't seen it, or they don't know.
We are humans, not a robot... --'

 

[Guilhermesene]

We are humans, not a robot...

Except for our great Robot @RoboMan, father of all robots, destroyer of all chains, slayer of all dragons...

 

[RoboMan]

Except for our great Robot @RoboMan, father of all robots, destroyer of all chains, slayer of all dragons...

I can confirm.

Regarding OP's question, I think that's a good question for our developers that are active in this forums, like @danb or @Andy Ful

 

[Shadowra]

Except for our great Robot @RoboMan, father of all robots, destroyer of all chains, slayer of all dragons...

Dragon ? Bitdefender ?

 

[Deletedmessiah]

@struppigel might be the best one around to ask since he works for G Data.

 

[Venustus]

And no response..... What a community.

I'm sure someone will answer your query in time

 

[struppigel]

What database engine AV companies uses for their malware database. "sqlite, Access, Excell , csv, realm" or something else ?

What exactly do you mean with malware database?
Are you referring to malware signatures or blocklists in the engine?
Or is this a question about backend systems?
And why is this important?

 

[Mr.NoName]

In my view, I think it is difficult to know the exact answer to this, more likely a Mysql, PostgreSQL or some database that focuses on Security without hindering performance.

That is the interesting what is this magical database

 

[Mr.NoName]

What exactly do you mean with malware database?
Are you referring to malware signatures or blocklists in the engine?
Or is this a question about backend systems?
And why is this important?

Virus stand alone database that is loaded into the usher system when he install let's say G-Data. Because as i know you can't depend only on cloud engine.... You need to have database inside the product with regular signature updates.

 

[LeMinhThanh]

What an interesting topic! Hope to learn more from this topic

 

[struppigel]

Virus stand alone database that is loaded into the usher system when he install let's say G-Data. Because as i know you can't depend only on cloud engine.... You need to have database inside the product with regular signature updates.

So you are referring to the signature database on the client with the patterns, hashes, and scripts that are checked by the scanning engine?
Those are usually custom-made formats to optimize performance and RAM usage and also to avoid making it obvious for malware devs how the signatures look like.

Edit: If you want to see an open source example, check out Yara and its compiled signature database.

 

[Mr.NoName]

So you are referring to the signature database on the client with the patterns, hashes, and scripts that are checked by the scanning engine?
Those are usually custom-made formats to optimize performance and RAM usage and also to avoid making it obvious for malware devs how the signatures look like.

Edit: If you want to see an open source example, check out Yara and its compiled signature database.

Yes i know yara and i am working with it. the interesting part is the to store this rules or hashes in db... After runing a few test's with app that i build with SQLITE in memory db and about 30 million hashes i get fast reading and writing but the cpu performace was even higher than read write time ... About 25 to 50% of Amd ryzen 16 thread's cpu. So my question is if you can give us some advice how to make our own format just to see if there is going to be a difference between Sqlite and costume made format.

 

[struppigel]

Yes i know yara and i am working with it. the interesting part is the to store this rules or hashes in db... After runing a few test's with app that i build with SQLITE in memory db and about 30 million hashes i get fast reading and writing but the cpu performace was even higher than read write time ... About 25 to 50% of Amd ryzen 16 thread's cpu. So my question is if you can give us some advice how to make our own format just to see if there is going to be a difference between Sqlite and costume made format.

I am a malware analyst. In my daily work I program mostly just short scripts for malware deobfuscation and I have not much to do with engine development. So this is not something I am experienced in.

 

[Andy Ful]

So my question is if you can give us some advice how to make our own format just to see if there is going to be a difference between Sqlite and costume made format.

You probably asked the wrong question. It is not especially important what format has the database as a file on disk. A more important question is how the data from this file looks in the memory (RAM) and how the AV access the data from this memory. Interesting information about how AVs can manage such data can be found somewhere on the web, for example: US8745743B2 - Anti-virus trusted files database - Google Patents

 

[Shadowra]

I am a malware analyst. In my daily work I program mostly just short scripts for malware deobfuscation and I have not much to do with engine development. So this is not something I am experienced in.

Malware analyst ? For which antivirus if you can tell?

 

[Gandalf_The_Grey]

Malware analyst ? For which antivirus if you can tell?

Just look at his profile:

Hello. My name is Karsten. I have been working as malware analyst for G DATA since 2015. This is a private account, though, I don't represent my employer.