Advice Request AV DB Engine

Please provide comments and solutions that are helpful to the author of this topic.

Kongo

Level 35
Verified
Top Poster
Well-known
Feb 25, 2017
2,481
And no response..... What a community.
cry-sad.gif
 

struppigel

Moderator
Verified
Staff Member
Well-known
Apr 9, 2020
656
What database engine AV companies uses for their malware database. "sqlite, Access, Excell :D, csv, realm" or something else ?
What exactly do you mean with malware database?
Are you referring to malware signatures or blocklists in the engine?
Or is this a question about backend systems?
And why is this important?
 

Mr.NoName

Level 4
Thread author
Verified
Feb 5, 2016
163
What exactly do you mean with malware database?
Are you referring to malware signatures or blocklists in the engine?
Or is this a question about backend systems?
And why is this important?
Virus stand alone database that is loaded into the usher system when he install let's say G-Data. Because as i know you can't depend only on cloud engine.... You need to have database inside the product with regular signature updates.
 

struppigel

Moderator
Verified
Staff Member
Well-known
Apr 9, 2020
656
Virus stand alone database that is loaded into the usher system when he install let's say G-Data. Because as i know you can't depend only on cloud engine.... You need to have database inside the product with regular signature updates.

So you are referring to the signature database on the client with the patterns, hashes, and scripts that are checked by the scanning engine?
Those are usually custom-made formats to optimize performance and RAM usage and also to avoid making it obvious for malware devs how the signatures look like.

Edit: If you want to see an open source example, check out Yara and its compiled signature database.
 

Mr.NoName

Level 4
Thread author
Verified
Feb 5, 2016
163
So you are referring to the signature database on the client with the patterns, hashes, and scripts that are checked by the scanning engine?
Those are usually custom-made formats to optimize performance and RAM usage and also to avoid making it obvious for malware devs how the signatures look like.

Edit: If you want to see an open source example, check out Yara and its compiled signature database.
Yes i know yara and i am working with it. the interesting part is the to store this rules or hashes in db... After runing a few test's with app that i build with SQLITE in memory db and about 30 million hashes i get fast reading and writing but the cpu performace was even higher than read write time :D... About 25 to 50% of Amd ryzen 16 thread's cpu. So my question is if you can give us some advice how to make our own format just to see if there is going to be a difference between Sqlite and costume made format.
 
  • Like
Reactions: Guilhermesene

struppigel

Moderator
Verified
Staff Member
Well-known
Apr 9, 2020
656
Yes i know yara and i am working with it. the interesting part is the to store this rules or hashes in db... After runing a few test's with app that i build with SQLITE in memory db and about 30 million hashes i get fast reading and writing but the cpu performace was even higher than read write time :D... About 25 to 50% of Amd ryzen 16 thread's cpu. So my question is if you can give us some advice how to make our own format just to see if there is going to be a difference between Sqlite and costume made format.
I am a malware analyst. In my daily work I program mostly just short scripts for malware deobfuscation and I have not much to do with engine development. So this is not something I am experienced in.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,042
So my question is if you can give us some advice how to make our own format just to see if there is going to be a difference between Sqlite and costume made format.
You probably asked the wrong question. It is not especially important what format has the database as a file on disk. A more important question is how the data from this file looks in the memory (RAM) and how the AV access the data from this memory. Interesting information about how AVs can manage such data can be found somewhere on the web, for example: US8745743B2 - Anti-virus trusted files database - Google Patents
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top