Reply to thread

Message

<blockquote data-quote="Andy Ful" data-source="post: 979465" data-attributes="member: 32260">The sad thing is that the &quot;EXE test error&quot; can be different for different AVs and can also depend on some random factors related to spam campaigns, targetted attacks, the choice of samples, etc. Furthermore, it is kind of a systematic error, so increasing the number of tests cannot improve the results.We can try to roughly estimate how big it can be.The payloads come from non-EXE initial attacks. Let&#039;s assume that:<ol>
<li data-xf-list-type="ol">Payloads are 50% of all tested samples (probably more).</li>
<li data-xf-list-type="ol">Antivirus &quot;A&quot; can block in the wild 30% more scripts and documents than antivirus &quot;B&quot;.</li>
<li data-xf-list-type="ol">Each of these additionally blocked scripts and documents could use one EXE payload (if not blocked). In real attacks, there can be more than one EXE payload, but sometimes there will be none. These payloads are unknown &quot;false payloads&quot; for antivirus &quot;A&quot;, because they were not used against this antivirus in the wild.</li>
<li data-xf-list-type="ol">About 2/3 of these fresh &amp; unknown &quot;false payloads&quot; could be detected/blocked by the antivirus &quot;A&quot;, and 1/3 could infect the computers.</li>
<li data-xf-list-type="ol">About 1/2 of these undetected &quot;false payloads&quot; could be added before the test to the detection basis via threat hunting and borrowing the detections from other AVs. This can be the main source of differences between AVs. Some vendors are slow and some much faster in adding detections for &quot;false payloads&quot;. The slow vendors will get greater &quot;EXE test error&quot; as compared to fast vendors.</li>
<li data-xf-list-type="ol">The chances of reusing the payload as an initial malware are negligible. This assumption is not true when lateral movement is an important attack vector or flash drives are extensively used for sharing files, cracks, pirated software. See also: 
[URL unfurl=&quot;false&quot;]https://malwaretips.com/threads/pecuriality-of-exe-malware-testing.112854/post-979543[/URL]</li>
</ol>Rate of false infections = 0.5 * 0.3 * 1/3 * 1/2 = 15/600 ~ 2.5%Of course, this calculation is only an example. But it shows that the testing error can hit a few percent.&nbsp; Furthermore, it does not apply to AVs that use file reputation lookup (the factor 1/3 must be replaced by something close to 0).Edit.Added point 6.</blockquote>

[QUOTE="Andy Ful, post: 979465, member: 32260"] The sad thing is that the "EXE test error" can be different for different AVs and can also depend on some random factors related to spam campaigns, targetted attacks, the choice of samples, etc. Furthermore, it is kind of a systematic error, so increasing the number of tests cannot improve the results. We can try to roughly estimate how big it can be. The payloads come from non-EXE initial attacks. Let's assume that: [LIST=1] [*]Payloads are 50% of all tested samples (probably more). [*]Antivirus "A" can block in the wild 30% more scripts and documents than antivirus "B". [*]Each of these additionally blocked scripts and documents could use one EXE payload (if not blocked). In real attacks, there can be more than one EXE payload, but sometimes there will be none. These payloads are unknown "false payloads" for antivirus "A", because they were not used against this antivirus in the wild. [*]About 2/3 of these fresh & unknown "false payloads" could be detected/blocked by the antivirus "A", and 1/3 could infect the computers. [*]About 1/2 of these undetected "false payloads" could be added before the test to the detection basis via threat hunting and borrowing the detections from other AVs. This can be the main source of differences between AVs. Some vendors are slow and some much faster in adding detections for "false payloads". The slow vendors will get greater "EXE test error" as compared to fast vendors. [*]The chances of reusing the payload as an initial malware are negligible. This assumption is not true when lateral movement is an important attack vector or flash drives are extensively used for sharing files, cracks, pirated software. See also: [URL unfurl="false"]https://malwaretips.com/threads/pecuriality-of-exe-malware-testing.112854/post-979543[/URL] [/LIST] [B]Rate of false infections = 0.5 * 0.3 * 1/3 * 1/2 = 15/600 ~ 2.5%[/B] Of course, this calculation is only an example. But it shows that the testing error can hit a few percent. Furthermore, it does not apply to AVs that use file reputation lookup (the factor 1/3 must be replaced by something close to 0). Edit. Added point 6. [/QUOTE]

Verification