Reply to thread

Message

<blockquote data-quote="Andy Ful" data-source="post: 906206" data-attributes="member: 32260">What is wrong with this chart?[ATTACH=full]246562[/ATTACH]The chart suggests that in Q2 2020 the in-the-wild detection of Bitdefender could be better as compared to for example Avast and can make the Avast users unhappy. <img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite111" alt=":(" title="Frown&nbsp; &nbsp; :(" loading="lazy" data-shortname=":(" />In fact, this suggestion could be valid only when all in-the-wild malware that might be missed by Bittedender had to be also missed by Avast.But, the sets of all missed in-the-wild malware for Avast and for Bitdefender are usually very different. So, by choosing 360 samples from tenths of thousands of different malware variants, we can easily miss more undetected samples of Avast than Bitdefender (and vice versa). In this way, the result would depend not on the real AV detection, but simply on the choice of the tested samples. One has to apply a statistical model to see if the data shows any advantage for Bitdefender. Unfortunately by doing this, it is easy to show that any of the first 6 AVs has no advantage. They have to be treated as they had the same detection in-the-wild. This model shows also that the first 6 AVs have a statistical advantage (in malware detection) over the last fourth. See also:[URL unfurl=&quot;true&quot;]https://malwaretips.com/threads/randomness-in-the-av-labs-testing.104104/[/URL][URL unfurl=&quot;true&quot;]https://malwaretips.com/threads/randomness-in-the-av-labs-testing.104104/post-905991[/URL]</blockquote>

[QUOTE="Andy Ful, post: 906206, member: 32260"] What is wrong with this chart? [ATTACH type="full" alt="AVC2020.png"]246562[/ATTACH] The chart suggests that in Q2 2020 the in-the-wild detection of Bitdefender could be better as compared to for example Avast and can make the Avast users unhappy. :( In fact, this suggestion could be valid only when [COLOR=rgb(184, 49, 47)][B]all in-the-wild malware that might be missed by Bittedender had to be also missed by Avast[/B][/COLOR]. But, the sets of all missed in-the-wild malware for Avast and for Bitdefender are usually very different. So, by choosing 360 samples from tenths of thousands of different malware variants, we can easily miss more undetected samples of Avast than Bitdefender (and vice versa). In this way, the result would depend not on the real AV detection, but simply on the choice of the tested samples. One has to apply a statistical model to see if the data shows any advantage for Bitdefender. Unfortunately by doing this, it is easy to show that any of the first 6 AVs has no advantage. They have to be treated as they had the same detection in-the-wild. This model shows also that the first 6 AVs have a statistical advantage (in malware detection) over the last fourth. See also: [URL unfurl="true"]https://malwaretips.com/threads/randomness-in-the-av-labs-testing.104104/[/URL] [URL unfurl="true"]https://malwaretips.com/threads/randomness-in-the-av-labs-testing.104104/post-905991[/URL] [/QUOTE]

Verification