Forums
New posts
Search forums
News
Security News
Technology News
Giveaways
Giveaways, Promotions and Contests
Discounts & Deals
Reviews
Users Reviews
Video Reviews
Support
Windows Malware Removal Help & Support
Inactive Support Threads
Mac Malware Removal Help & Support
Mobile Malware Removal Help & Support
Blog
Log in
Register
What's new
Search
Search titles only
By:
Search titles only
By:
Reply to thread
Menu
Install the app
Install
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Forums
Security
Malware Analysis
The peculiarity of EXE malware testing.
Message
<blockquote data-quote="Andy Ful" data-source="post: 979714" data-attributes="member: 32260"><p><strong><span style="font-size: 18px">YouTube tests and "EXE test error"</span></strong></p><p></p><p>My previous posts were rather technical and probably hard to understand. So, it would be useful to describe it in a simpler way.</p><p></p><p>Many people are convinced that the home-made AV tests on YouTube can reflect the true capabilities of AVs. They use thousands of few-days-old (or few-weeks-old) EXE samples and think that using so many samples makes their tests faithful. They also think that if the particular AV has not got a very good scoring in such tests, then it cannot protect very well against unknown 0-day EXE malware.</p><p></p><p>Unfortunately, they are wrong for several reasons. One of these reasons is "EXE test error". Such tests could be reliable when the samples were few-hours-old, but not few-days-old.</p><p><span style="font-size: 15px"><em>When one uses older EXE samples in testing, then the test shows simply how could be the protection if the attackers would bother to reuse older EXE payloads as initial malware, instead of creating new malware. So, the result of such a test is highly artificial. The attackers prefer to use new samples because these samples are far more effective.</em></span></p><p></p><p>Generally, the "EXE test error" says that AVs that do not have good protection against scripts and weaponized documents are favored (get undeservedly better scorings) compared to AVs that have got stronger protection/prevention against scripts and weaponized documents. <strong>This happens even when AVs have got the same capabilities to detect EXE files.</strong></p><p>The quantity of the "EXE test error" can also depend on how much the AV vendors care about the older samples that never hit their customers.</p><p>Anyway, it does not apply to AVs that use reputation file lookup for EXE files (or smart default-deny approach), because these AVs can protect against EXE files independently of the non-EXE protection layers.</p><p></p><p></p><p><strong><span style="font-size: 18px">Can the EXE tests be valuable?</span></strong></p><p></p><p><span style="font-size: 15px">Yes, they can. But not to compare the protective abilities against EXE malware (when few-days-old or older samples are used). </span></p><p><span style="font-size: 15px"></span></p><p><span style="font-size: 15px">The EXE tests can be still valuable in several ways:</span></p><ol> <li data-xf-list-type="ol"><span style="font-size: 15px">They can mimic the scenario when flash drives (USB drives) are used in an extensive way to share cracks, pirated software, etc.</span></li> <li data-xf-list-type="ol"><span style="font-size: 15px">For AV vendors to improve their products. </span></li> <li data-xf-list-type="ol">For people who want some more information about how AVs work.</li> <li data-xf-list-type="ol">As the demonstration of capabilities.</li> <li data-xf-list-type="ol">As a kind of support test to Real-World testing.</li> </ol><p>The "EXE + scripts" test can mimic the scenario of attacks via lateral movement in businesses and organizations.</p><p></p><p>Edit.</p><p>Slightly simplified the content.</p></blockquote><p></p>
[QUOTE="Andy Ful, post: 979714, member: 32260"] [B][SIZE=5]YouTube tests and "EXE test error"[/SIZE][/B] My previous posts were rather technical and probably hard to understand. So, it would be useful to describe it in a simpler way. Many people are convinced that the home-made AV tests on YouTube can reflect the true capabilities of AVs. They use thousands of few-days-old (or few-weeks-old) EXE samples and think that using so many samples makes their tests faithful. They also think that if the particular AV has not got a very good scoring in such tests, then it cannot protect very well against unknown 0-day EXE malware. Unfortunately, they are wrong for several reasons. One of these reasons is "EXE test error". Such tests could be reliable when the samples were few-hours-old, but not few-days-old. [SIZE=4][I]When one uses older EXE samples in testing, then the test shows simply how could be the protection if the attackers would bother to reuse older EXE payloads as initial malware, instead of creating new malware. So, the result of such a test is highly artificial. The attackers prefer to use new samples because these samples are far more effective.[/I][/SIZE] Generally, the "EXE test error" says that AVs that do not have good protection against scripts and weaponized documents are favored (get undeservedly better scorings) compared to AVs that have got stronger protection/prevention against scripts and weaponized documents. [B]This happens even when AVs have got the same capabilities to detect EXE files.[/B] The quantity of the "EXE test error" can also depend on how much the AV vendors care about the older samples that never hit their customers. Anyway, it does not apply to AVs that use reputation file lookup for EXE files (or smart default-deny approach), because these AVs can protect against EXE files independently of the non-EXE protection layers. [B][SIZE=5]Can the EXE tests be valuable?[/SIZE][/B] [SIZE=4]Yes, they can. But not to compare the protective abilities against EXE malware (when few-days-old or older samples are used). The EXE tests can be still valuable in several ways:[/SIZE] [LIST=1] [*][SIZE=4]They can mimic the scenario when flash drives (USB drives) are used in an extensive way to share cracks, pirated software, etc.[/SIZE] [*][SIZE=4]For AV vendors to improve their products. [/SIZE] [*]For people who want some more information about how AVs work. [*]As the demonstration of capabilities. [*]As a kind of support test to Real-World testing. [/LIST] The "EXE + scripts" test can mimic the scenario of attacks via lateral movement in businesses and organizations. Edit. Slightly simplified the content. [/QUOTE]
Insert quotes…
Verification
Post reply
Top
