AV-Comparatives Consumer Malware Protection Test September 2020

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Gandalf_The_Grey

Level 76
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 24, 2016
6,506
We released our Consumer Malware Protection Test. Any samples that have not been detected e.g. on-access are executed on the test system. A false alarm test is also included. While in the Real-World Protection Test the vector is the web, in the Malware Protection Test the vectors can be e.g. network drives, USB or cover scenarios where the malware is already on the disk.

Independent Austrian antivirus testing lab AV-Comparatives has released the report of its September 2020 Malware Protection Test for consumer security programs. 17 popular anti-malware apps were put through their paces. To ensure that the tested programs do not protect the system at the expense of high false-alarm rates, a false-positives test is also run. The report shows both the protection and false-positive rates, and can be downloaded free from the AV-Comparatives website.
Malware Protection Test September 2020:
False Alarm Test September 2020:
 

EndangeredPootis

Level 10
Verified
Well-known
Sep 8, 2019
461
What's the chance 4 will all have 99.96%, 3 with 99.98%, 3 with 99.9% detection ratio at the same time? exactly, it makes no sense, and I still don't understand why people still trust in these test sites, especially considering the fact they never show them taking place, it is like believing in a president or king, just because many people support7trust them doesn't mean you can trust everything they say.

Also, they gave WD 2 stars despite it having supposedley 100% detection ratio, for what? it had an higher than average false positive rate? how does that make sense, it can easily mislead people.
 
Last edited:

SeriousHoax

Level 47
Well-known
Mar 16, 2019
3,630
This test shows,
Microsoft/Windows Defender just keeps getting better.
Bitdefender may not be as fast as they used to be creating signatures but they are still the AV that makes the most.
Trend Micro is a ridiculously cloud dependent AV and compromised 175 malwares.
Avast/AVG are maybe nagger and privacy nightmare but they remain a top AV.

I won't pay too much attention on the false positives number as you can see, most of those are app packages that you probably have never even heard of. @MacDefender explained it really well in one of the last result threads posted here on MT.
 
L

Local Host

What's the chance 4 will all have 99.96%, 3 with 99.98%, 3 with 99.9% detection ratio at the same time? exactly, it makes no sense, and I still don't understand why people still trust in these test sites, especially considering the fact they never show them taking place, it is like believing in a president or king, just because many people support7trust them doesn't mean you can trust everything they say.

Also, they gave WD 2 stars despite it having supposedley 100% detection ratio, for what? it had an higher than average false positive rate? how does that make sense, it can easily mislead people.
It makes total sense, since this is a consumer test, most malware that reaches home users is known.

The same way it makes sense for WD to have only 2 stars, as briefly explained in the test, false positives serve to prove how effective the AV is.

WD detected 100% cause it pretty much saw everything as malware (high false positives), so is pretty clear it doesn't deserve to pass this test (it didn't legit detect the malware like the others).
 

plat

Level 29
Top Poster
Sep 13, 2018
1,793
Well...McAfee brand is kind of respectable nowadays. It used to rank near- or at the bottom competing with the major ones, that's true. I see its performance in the Hub now and then; I mean, you could do worse. AVs can be "allowed" to exceed expectations every once in a while, as long as the test is conducted on the up-and-up.

The founder of McAfee, however, continues to be kind of a nut job in my opinion. A weird-o. Also, he was a "person of interest" in a murder a while back, remember? Now he's charged with tax evasion. lol!
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,042
It is worth to look at the Cluster column in the scoring table:

1602801484714.png


It seems that cluster analysis includes the natural randomness of results that follows from the testing procedure. The test cannot prove any difference in the real protection of the first 11 AVs (they belong to the same cluster).(y)
The cluster analysis gives similar results as the statistical model described in the thread "Randomness in the AV Labs testing", for example:
 
Last edited:

upnorth

Moderator
Verified
Staff Member
Malware Hunter
Well-known
Jul 27, 2015
5,459
The truth is this: AV tests are marketing tools. Nothing else. Picking 100 samples per month out of a pool of tens of millions will tell you absolutely nothing about the effectiveness of an AV. Quite frankly: The malware hub here in this forum probably tests with more samples than AV-Test or AV-C do every month for their "real world protection" tests.
 

shmu26

Level 85
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Jul 3, 2015
8,150
IMHO, Bitdefender did well, can't win every single time, but 99.9% is quite good I would think. ;)
I am very impressed by the Bitdefender offline detection! Sometimes your internet connection is flakey, or the server of your AV is offline or overloaded. So offline protection is important.
As for online protection, the difference between 99.9% and 100% is totally negligible.
As far as I am concerned, Bitdefender is the winner in protection.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,042
AV-Comparatives Malware Protection tests 2019-2020.

Cumulative results for last 2 years (March 2019, September 2019, March 2020, September 2020):

----------------Missed samples----Clusters
Group 1
Avast, AVG........1+0+2+0.............1,1,1,1
Norton...............(2)+(2)+0+2.......1,1,1,1
Bitdefender.......1+5+2+1.............1,1,1,1
K7......................5+5+1+2.............1,1,1,1
Avira* ................0+4+3+4............1,1,1,2

Group 2
Panda ................1+1+4+11.........1,1,1,3
Microsoft...........2+4+12+0..........1,1,3,1
VIPRE ................4+10+3+4 .........1,2,1,2
F-Secure ...........14+9+0+1.......... 2,2,1,1
Kaspersky......... 13+9+3+1..........2,2,1,1
ESET...................15+12+1+2 .......2,2,1,1
Total Defense*...5+19+1+4.........1,3,1,2

Group 3
McAfee ............11+19+7+0.........2,3,2,1

Hard to classify
Trend Micro......0+0+82+175...... 1,1,4,4

The last column shows cluster analysis, which is most important. The number of missed samples is less important, but I have taken it as the secondary classification parameter (Avira, TotalDefense).
These tests cannot prove what is real protection of AVs in the same group (cluster analysis) - they can have the same protection in the wild despite a different number of missed samples.
Furthermore, the average detection difference between Group 1 and 2 is very little ( about 17 missed samples per 42000 total samples).
So, in practice, it would be hard to see the protection difference between most of the tested AVs (which is already a well known fact).:)(y)

 
Last edited:

plat

Level 29
Top Poster
Sep 13, 2018
1,793
Btw, he's not part of McAfee anymore. He sold it more than 20 years ago.

Yes, right, he still founded the company. I looked up the Wiki on McAfee; it's kind of amusing in spots. When Intel changed the name "McAfee" to Intel Security, he reportedly said: "I am now everlastingly grateful to Intel for freeing me from this terrible association with the worst software on the planet....."

What surprised me from that Wiki was that Intel still owns a 49% stake and Thoma Bravo, owner of Sophos, owns a minor stake also.

Source
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top