Forums
New posts
Search forums
News
Security News
Technology News
Giveaways
Giveaways, Promotions and Contests
Discounts & Deals
Reviews
Users Reviews
Video Reviews
Support
Windows Malware Removal Help & Support
Inactive Support Threads
Mac Malware Removal Help & Support
Mobile Malware Removal Help & Support
Blog
Log in
Register
What's new
Search
Search titles only
By:
Search titles only
By:
Reply to thread
Menu
Install the app
Install
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Forums
Security
General Security Discussions
A Brief Critique of Professional AV Tests
Message
<blockquote data-quote="Adrian Ścibor" data-source="post: 1037937" data-attributes="member: 71496"><p>Interesting topic... I have just already watching a Matrix trilogy once again, and I felt bored, so I switched to MalwareTips news or something and voilà! The real game is better! <img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite109" alt=":)" title="Smile :)" loading="lazy" data-shortname=":)" /></p><p></p><p>Backing to the point and video with a quote from AVLab's methodology. I need to add a full description of how we chose a malware file for testing:</p><p></p><p>[URL unfurl="true"]https://avlab.pl/en/methodology/[/URL]</p><p></p><p>We used malware source in the wild:</p><ul> <li data-xf-list-type="ul"><a href="https://github.com/CERT-Polska/mwdb-core" target="_blank">MWDB</a> project by CERT Poland.</li> <li data-xf-list-type="ul"><a href="https://bazaar.abuse.ch/" target="_blank">Malware Bazaar</a> project by Abuse.ch.</li> <li data-xf-list-type="ul">Starting from May, we will use public URLs from <a href="https://urlquery.net/" target="_blank">urlquery.net - Automated URL scanner</a></li> <li data-xf-list-type="ul">Custom honeypots based on <a href="https://github.com/DinoTools/dionaea" target="_blank">Dionaea</a> -> but this is not a good source, because there are a lot of samples, but a lot is repeated, duplicated of SHA.</li> <li data-xf-list-type="ul">We can use additional sources, if you can help us with that, then we can include and automate to our tests. We tried, for example, with app.any.run, but they do not have the right API to get only URLs.</li> </ul><p>Generally, the whole industry used something like WildList, which is not being developed. At AMTSO we are working on replacing it with own list, but it's not easy. It requires a lot of people and often for free to build something like this. AMTSO currently has its own RTTL list, but we do not use this.</p><p></p><p>What is going on next?</p><p></p><p>1. Every URL potentially contains something that can be downloaded. We download it and check it (mal score.png)</p><p></p><p>a. a file is scanning by Linux tool for matching file type and duplication of SHA256 in database (if duplicate, it is rejected and starting new queue)</p><p>b. a file is scanning by some Yara rules:</p><ul> <li data-xf-list-type="ul">Rules included in packer_compiler_signatures.yar to detect broken or damaged portable executable files.</li> <li data-xf-list-type="ul">Rules included in maldocs_index.yar to detect good or bad files with macros contained in Microsoft Office.</li> <li data-xf-list-type="ul">Rules included in anti_sandboxing.yar to detect anti-vm techniques that prevent from executing in virtual environment of Windows.</li> </ul><p>c. only after all, the sample runs in the black box (Windows, without AV protection) to check potential malicious changes based on Sysmon rules and logs - as mentioned by the author of the thread.</p><p></p><p>If the potential file is "good" based on point C, a parameter from the URL is passed to the browser to all machines with security products installed. From this point onwards, the malware is analyzed at the same time and the response of the security software is checked.</p><p></p><p>In addition, after the May edition, we'll publish an external CSV with 3rd party scanner opinion about malware. This is implemented and it's not a secret. The technology provider is Arcabit/MKS_VIR from Poland. We do not test it, so I do not see a conflict of interest here. We will add these data to our <a href="https://avlab.pl/en/changelog/" target="_blank">changelog</a> soon.</p><p></p><p>Hopefully, this will further exclude usage of potentially useless, non-malware samples in AVLab tests (Advanced In The Wild Malware Test).</p><p></p><p>#####</p><p>If there is a willingness on your part to be interested, I can make a video for you of how it all works in turn from the inside.</p></blockquote><p></p>
[QUOTE="Adrian Ścibor, post: 1037937, member: 71496"] Interesting topic... I have just already watching a Matrix trilogy once again, and I felt bored, so I switched to MalwareTips news or something and voilà! The real game is better! :) Backing to the point and video with a quote from AVLab's methodology. I need to add a full description of how we chose a malware file for testing: [URL unfurl="true"]https://avlab.pl/en/methodology/[/URL] We used malware source in the wild: [LIST] [*][URL='https://github.com/CERT-Polska/mwdb-core']MWDB[/URL] project by CERT Poland. [*][URL='https://bazaar.abuse.ch/']Malware Bazaar[/URL] project by Abuse.ch. [*]Starting from May, we will use public URLs from [URL='https://urlquery.net/']urlquery.net - Automated URL scanner[/URL] [*]Custom honeypots based on [URL='https://github.com/DinoTools/dionaea']Dionaea[/URL] -> but this is not a good source, because there are a lot of samples, but a lot is repeated, duplicated of SHA. [*]We can use additional sources, if you can help us with that, then we can include and automate to our tests. We tried, for example, with app.any.run, but they do not have the right API to get only URLs. [/LIST] Generally, the whole industry used something like WildList, which is not being developed. At AMTSO we are working on replacing it with own list, but it's not easy. It requires a lot of people and often for free to build something like this. AMTSO currently has its own RTTL list, but we do not use this. What is going on next? 1. Every URL potentially contains something that can be downloaded. We download it and check it (mal score.png) a. a file is scanning by Linux tool for matching file type and duplication of SHA256 in database (if duplicate, it is rejected and starting new queue) b. a file is scanning by some Yara rules: [LIST] [*]Rules included in packer_compiler_signatures.yar to detect broken or damaged portable executable files. [*]Rules included in maldocs_index.yar to detect good or bad files with macros contained in Microsoft Office. [*]Rules included in anti_sandboxing.yar to detect anti-vm techniques that prevent from executing in virtual environment of Windows. [/LIST] c. only after all, the sample runs in the black box (Windows, without AV protection) to check potential malicious changes based on Sysmon rules and logs - as mentioned by the author of the thread. If the potential file is "good" based on point C, a parameter from the URL is passed to the browser to all machines with security products installed. From this point onwards, the malware is analyzed at the same time and the response of the security software is checked. In addition, after the May edition, we'll publish an external CSV with 3rd party scanner opinion about malware. This is implemented and it's not a secret. The technology provider is Arcabit/MKS_VIR from Poland. We do not test it, so I do not see a conflict of interest here. We will add these data to our [URL='https://avlab.pl/en/changelog/']changelog[/URL] soon. Hopefully, this will further exclude usage of potentially useless, non-malware samples in AVLab tests (Advanced In The Wild Malware Test). ##### If there is a willingness on your part to be interested, I can make a video for you of how it all works in turn from the inside. [/QUOTE]
Insert quotes…
Verification
Post reply
Top
