Forums
New posts
Search forums
News
Security News
Technology News
Giveaways
Giveaways, Promotions and Contests
Discounts & Deals
Reviews
Users Reviews
Video Reviews
Support
Windows Malware Removal Help & Support
Inactive Support Threads
Mac Malware Removal Help & Support
Mobile Malware Removal Help & Support
Blog
Log in
Register
What's new
Search
Search titles only
By:
Search titles only
By:
Reply to thread
Menu
Install the app
Install
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Forums
Security
Malware Analysis
The peculiarity of EXE malware testing.
Message
<blockquote data-quote="Andy Ful" data-source="post: 979614" data-attributes="member: 32260"><p><strong><span style="font-size: 18px">Is Defender better than TrendMicro?</span></strong></p><p></p><p>Here are the results of AVLab tests from the years 2019-2021. I skipped the test from January 2022 because the results for Defender (only L3 level detections) contradict the AVLab testing methodology.</p><p></p><p>............................MONTH:.. J......S.....O...N....j....m...M...J....S....N....j...m...M...J....S....N..</p><p><strong>Defender ............................ </strong>x<strong> ... </strong>x<strong> ...<span style="color: rgb(41, 105, 176)">17</span>.. <span style="color: rgb(41, 105, 176)">0</span> .. </strong>x<strong>.. <span style="color: rgb(41, 105, 176)">20</span>..</strong> x<strong>... </strong>x.<strong>.. <span style="color: rgb(41, 105, 176)">0</span>... </strong>x<strong>... <span style="color: rgb(41, 105, 176)">8</span>... <span style="color: rgb(41, 105, 176)">0</span>... <span style="color: rgb(41, 105, 176)">0</span> ...</strong>x<strong>... <span style="color: rgb(41, 105, 176)">2</span>... </strong>x<strong> = <span style="color: rgb(41, 105, 176)">47</span></strong></p><p><strong>TrendMicro ........................ </strong>x<strong> ... </strong>x .<strong>.. </strong>x<strong> ... </strong>x<strong> ... </strong>x<strong>.. <span style="color: rgb(41, 105, 176)">2</span>..<span style="color: rgb(41, 105, 176)">158</span> </strong>x<strong> ... </strong>x<strong> ...</strong>x<strong> ...</strong>x<strong> ...</strong>x<strong> ...</strong>x<strong> ...</strong>x<strong> ...</strong>x <strong>... </strong>x<strong> = <span style="color: rgb(41, 105, 176)">160</span></strong></p><p></p><p>TrendMicro missed more samples in 2 tests than Defender in 8 tests. So the result for TrendMicro is much worse.</p><p>But, does it mean that TrendMicro has got much worse protection against the EXE files? I do not think so. In such a situation, TrendMicro could not get top results (better than Defender free) in the Real-World tests. So what is happening here?</p><p></p><p>Defender is used by the customers as a free AV and in Enterprises as a paid AV. Defender free on default settings has not got such strong protection/prevention against weaponized documents as TrendMicro (not sure about scripts). It also seems that Microsoft can more carefully add detections of "false payloads". So, the much better result of Defender (113 fewer missed samples) is not real and probably caused by "EXE test error". </p><p></p><p>I think that "EXE test error" can be also responsible for the results in AV-Comparatives Malware Protection tests.</p><p>Malware Protection 2019-2021, missed samples:</p><p>Microsoft........<span style="color: rgb(41, 105, 176)"><strong>36</strong></span></p><p>TrendMicro...<span style="color: rgb(41, 105, 176)"><strong>481</strong></span></p><p>Samples.....61 919</p><p></p><p>The "EXE test error" (about 445 samples per 61919) is in fact relatively small ~ 0.7% . This can be related to using prevalent samples two-weeks-old (on average). In the case of AV-Comparatives, the samples were prevalent, so it is probable that Defender free missed in the wild most of these 445 payloads (via undetected documents), when they were 0-day or 1-day old samples (before the test).</p><p></p><p>Post edited.</p></blockquote><p></p>
[QUOTE="Andy Ful, post: 979614, member: 32260"] [B][SIZE=5]Is Defender better than TrendMicro?[/SIZE][/B] Here are the results of AVLab tests from the years 2019-2021. I skipped the test from January 2022 because the results for Defender (only L3 level detections) contradict the AVLab testing methodology. ............................MONTH:.. J......S.....O...N....j....m...M...J....S....N....j...m...M...J....S....N.. [B]Defender ............................ [/B]x[B] ... [/B]x[B] ...[COLOR=rgb(41, 105, 176)]17[/COLOR].. [COLOR=rgb(41, 105, 176)]0[/COLOR] .. [/B]x[B].. [COLOR=rgb(41, 105, 176)]20[/COLOR]..[/B] x[B]... [/B]x.[B].. [COLOR=rgb(41, 105, 176)]0[/COLOR]... [/B]x[B]... [COLOR=rgb(41, 105, 176)]8[/COLOR]... [COLOR=rgb(41, 105, 176)]0[/COLOR]... [COLOR=rgb(41, 105, 176)]0[/COLOR] ...[/B]x[B]... [COLOR=rgb(41, 105, 176)]2[/COLOR]... [/B]x[B] = [COLOR=rgb(41, 105, 176)]47[/COLOR] TrendMicro ........................ [/B]x[B] ... [/B]x .[B].. [/B]x[B] ... [/B]x[B] ... [/B]x[B].. [COLOR=rgb(41, 105, 176)]2[/COLOR]..[COLOR=rgb(41, 105, 176)]158[/COLOR] [/B]x[B] ... [/B]x[B] ...[/B]x[B] ...[/B]x[B] ...[/B]x[B] ...[/B]x[B] ...[/B]x[B] ...[/B]x [B]... [/B]x[B] = [COLOR=rgb(41, 105, 176)]160[/COLOR][/B] TrendMicro missed more samples in 2 tests than Defender in 8 tests. So the result for TrendMicro is much worse. But, does it mean that TrendMicro has got much worse protection against the EXE files? I do not think so. In such a situation, TrendMicro could not get top results (better than Defender free) in the Real-World tests. So what is happening here? Defender is used by the customers as a free AV and in Enterprises as a paid AV. Defender free on default settings has not got such strong protection/prevention against weaponized documents as TrendMicro (not sure about scripts). It also seems that Microsoft can more carefully add detections of "false payloads". So, the much better result of Defender (113 fewer missed samples) is not real and probably caused by "EXE test error". I think that "EXE test error" can be also responsible for the results in AV-Comparatives Malware Protection tests. Malware Protection 2019-2021, missed samples: Microsoft........[COLOR=rgb(41, 105, 176)][B]36[/B][/COLOR] TrendMicro...[COLOR=rgb(41, 105, 176)][B]481[/B][/COLOR] Samples.....61 919 The "EXE test error" (about 445 samples per 61919) is in fact relatively small ~ 0.7% . This can be related to using prevalent samples two-weeks-old (on average). In the case of AV-Comparatives, the samples were prevalent, so it is probable that Defender free missed in the wild most of these 445 payloads (via undetected documents), when they were 0-day or 1-day old samples (before the test). Post edited. [/QUOTE]
Insert quotes…
Verification
Post reply
Top
