Reply to thread

Message

<blockquote data-quote="Andy Ful" data-source="post: 1116058" data-attributes="member: 32260">Post updated.Here is an example of how the test delay could impact the protection rate of many AVs.In the example, I assumed that when the home users run the malware, only 1 per 10 malware is unknown to the AV (70% samples detected/blocked) and 9 per 10 malware is already known (100% samples detected/blocked): 1/10 * 70% + 9/10 * 100% = 97%&nbsp; &nbsp;&nbsp; &lt;-------&nbsp; protection rate in the wild1/10 * 95% + 9/10 * 100% = 99.5%&nbsp; &nbsp; &lt;-------&nbsp; protection rate in the Real-World testAfter the delay, many unknown samples &quot;become better known&quot; to AV. The protection rate for the previously unknown samples increases to 95% (instead of the initial 70%), and the final result is similar to the Real-World tests.Edit1.By &quot;become better known,&quot; I mean that due to test delay, some previously unknown samples can now be detected/blocked by possible improvements of ML classifiers or new signatures.For simplicity, I skipped the samples that became inactive during the delay time. After including them, the difference between the AV protection in the wild and the protection shown in the Real-World tests can be even greater.Edit2.Using Application Control, file reputation lookup, or detonation in the sandbox can significantly increase AV protection. Some AVs (like Norton or Avast) use such features by default, some others require tweaking the settings. On Windows 8+, the file reputation lookup (for files originating from the Internet) can be applied via the Windows built-in SmartScreen.</blockquote>

[QUOTE="Andy Ful, post: 1116058, member: 32260"] Post updated. Here is an example of how the test delay could impact the protection rate of many AVs. In the example, I assumed that when the home users run the malware, only 1 per 10 malware is unknown to the AV (70% samples detected/blocked) and 9 per 10 malware is already known (100% samples detected/blocked): [B] 1/10 * [COLOR=rgb(184, 49, 47)]70% [/COLOR]+ 9/10 *[COLOR=rgb(184, 49, 47)] 100% = 97% <------- protection rate in the wild[/COLOR] 1/10 *[COLOR=rgb(0, 168, 133)] 95% [/COLOR] + 9/10 * [COLOR=rgb(0, 168, 133)]100% = 99.5% <------- protection rate in the Real-World test[/COLOR][/B] After the delay, many unknown samples "become better known" to AV. The protection rate for the previously unknown samples increases to [COLOR=rgb(0, 168, 133)][B]95%[/B][/COLOR] (instead of the initial [COLOR=rgb(184, 49, 47)][B]70%[/B][/COLOR]), and the final result is similar to the Real-World tests. Edit1. By "become better known," I mean that due to test delay, some previously unknown samples can now be detected/blocked by possible improvements of ML classifiers or new signatures. For simplicity, I skipped the samples that became inactive during the delay time. After including them, the difference between the AV protection in the wild and the protection shown in the Real-World tests can be even greater. Edit2. Using Application Control, file reputation lookup, or detonation in the sandbox can significantly increase AV protection. Some AVs (like Norton or Avast) use such features by default, some others require tweaking the settings. On Windows 8+, the file reputation lookup (for files originating from the Internet) can be applied via the Windows built-in SmartScreen. [/QUOTE]

Verification