AV-Comparatives Consumer Malware Protection Test September 2020

[Gandalf_The_Grey]

We released our Consumer Malware Protection Test. Any samples that have not been detected e.g. on-access are executed on the test system. A false alarm test is also included. While in the Real-World Protection Test the vector is the web, in the Malware Protection Test the vectors can be e.g. network drives, USB or cover scenarios where the malware is already on the disk.

Independent Austrian antivirus testing lab AV-Comparatives has released the report of its September 2020 Malware Protection Test for consumer security programs. 17 popular anti-malware apps were put through their paces. To ensure that the tested programs do not protect the system at the expense of high false-alarm rates, a false-positives test is also run. The report shows both the protection and false-positive rates, and can be downloaded free from the AV-Comparatives website.

Malware Protection Test September 2020:

Malware Protection Test September 2020 - AV-Comparatives

The Malware Protection Test assesses a security program’s ability to protect a system against infection by malicious files before, during or after execution.

www.av-comparatives.org

False Alarm Test September 2020:

False Alarm Test September 2020 - AV-Comparatives

www.av-comparatives.org

 

[Bryan320]

windows defender came out on top

 

[Digmor Crusher]

Defender has been one of the top products tested for some time now.

 

[EndangeredPootis]

What's the chance 4 will all have 99.96%, 3 with 99.98%, 3 with 99.9% detection ratio at the same time? exactly, it makes no sense, and I still don't understand why people still trust in these test sites, especially considering the fact they never show them taking place, it is like believing in a president or king, just because many people support7trust them doesn't mean you can trust everything they say.

Also, they gave WD 2 stars despite it having supposedley 100% detection ratio, for what? it had an higher than average false positive rate? how does that make sense, it can easily mislead people.

 

[Bryan320]

Defender has been one of the top products tested for some time now.

i am aware of that however it blocked everything in the test. The false positives gave it the downgrade but to see it detect more than bitdefender and Kaspersky is surprising.

 

[SeriousHoax]

This test shows,
Microsoft/Windows Defender just keeps getting better.
Bitdefender may not be as fast as they used to be creating signatures but they are still the AV that makes the most.
Trend Micro is a ridiculously cloud dependent AV and compromised 175 malwares.
Avast/AVG are maybe nagger and privacy nightmare but they remain a top AV.

I won't pay too much attention on the false positives number as you can see, most of those are app packages that you probably have never even heard of. @MacDefender explained it really well in one of the last result threads posted here on MT.

 

[Local Host]

What's the chance 4 will all have 99.96%, 3 with 99.98%, 3 with 99.9% detection ratio at the same time? exactly, it makes no sense, and I still don't understand why people still trust in these test sites, especially considering the fact they never show them taking place, it is like believing in a president or king, just because many people support7trust them doesn't mean you can trust everything they say.

Also, they gave WD 2 stars despite it having supposedley 100% detection ratio, for what? it had an higher than average false positive rate? how does that make sense, it can easily mislead people.

It makes total sense, since this is a consumer test, most malware that reaches home users is known.

The same way it makes sense for WD to have only 2 stars, as briefly explained in the test, false positives serve to prove how effective the AV is.

WD detected 100% cause it pretty much saw everything as malware (high false positives), so is pretty clear it doesn't deserve to pass this test (it didn't legit detect the malware like the others).

 

[WhiteMouse]

It's funny how most AVs detect ESET package as trojan or riskware

 

[marcopaone]

AHHAAHHA
Gdata,Windows Defender and McAfee 100%

AHAHAHAHAHAH What a joke.

 

[plat1098]

Well...McAfee brand is kind of respectable nowadays. It used to rank near- or at the bottom competing with the major ones, that's true. I see its performance in the Hub now and then; I mean, you could do worse. AVs can be "allowed" to exceed expectations every once in a while, as long as the test is conducted on the up-and-up.

The founder of McAfee, however, continues to be kind of a nut job in my opinion. A weird-o. Also, he was a "person of interest" in a murder a while back, remember? Now he's charged with tax evasion. lol!

 

[SecureKongo]

AHHAAHHA
Gdata,Windows Defender and McAfee 100%

AHAHAHAHAHAH What a joke.

G-Data is quite decent actually.

 

[RoboMan]

Kaspersky with Application Control configured right would have been top

Glad for Microsoft! It's great they are offering solid protection to their users!

 

[Andy Ful]

It is worth to look at the Cluster column in the scoring table:



It seems that cluster analysis includes the natural randomness of results that follows from the testing procedure. The test cannot prove any difference in the real protection of the first 11 AVs (they belong to the same cluster).
The cluster analysis gives similar results as the statistical model described in the thread "Randomness in the AV Labs testing", for example:

AV-Comparatives - Randomness in the AV Labs testing.

Let's assume that we have the below results for the test: 2 AVs with 0 undetected malware 4 AVs with 1 undetected malware 4 AVs with 2 undetected malware 3 AVs with 3 undetected malware 1 AV with 4 undetected malware It can be very close to probabilities for hypothetical AV when: k = 60...

malwaretips.com

 

[jtshadow92]

Glad McAfee improved and scored 100%

 

[FireHammer]

IMHO, Bitdefender did well, can't win every single time, but 99.9% is quite good I would think.

 

[SeriousHoax]

The founder of McAfee, however, continues to be kind of a nut job in my opinion. A weird-o. Also, he was a "person of interest" in a murder a while back, remember? Now he's charged with tax evasion. lol!

Btw, he's not part of McAfee anymore. He sold it more than 20 years ago.

 

[upnorth]

The truth is this: AV tests are marketing tools. Nothing else. Picking 100 samples per month out of a pool of tens of millions will tell you absolutely nothing about the effectiveness of an AV. Quite frankly: The malware hub here in this forum probably tests with more samples than AV-Test or AV-C do every month for their "real world protection" tests.

Q&A - I am head of research at Emsisoft. Ask me anything! :)

Telemetry is important and useful, for any company. We started collecting telemetry as well (although nowhere near as some other companies do). Is it considered private information if there is a record which buttons were clicked how many times? I would argue not. So telemetry on its own isn't...

malwaretips.com

 

[shmu26]

IMHO, Bitdefender did well, can't win every single time, but 99.9% is quite good I would think.

I am very impressed by the Bitdefender offline detection! Sometimes your internet connection is flakey, or the server of your AV is offline or overloaded. So offline protection is important.
As for online protection, the difference between 99.9% and 100% is totally negligible.
As far as I am concerned, Bitdefender is the winner in protection.

 

[Andy Ful]

AV-Comparatives Malware Protection tests 2019-2020.

Cumulative results for last 2 years (March 2019, September 2019, March 2020, September 2020):

----------------Missed samples----Clusters
Group 1
Avast, AVG........1+0+2+0.............1,1,1,1
Norton...............(2)+(2)+0+2.......1,1,1,1
Bitdefender.......1+5+2+1.............1,1,1,1
K7......................5+5+1+2.............1,1,1,1
Avira* ................0+4+3+4............1,1,1,2

Group 2
Panda ................1+1+4+11.........1,1,1,3
Microsoft...........2+4+12+0..........1,1,3,1
VIPRE ................4+10+3+4 .........1,2,1,2
F-Secure ...........14+9+0+1.......... 2,2,1,1
Kaspersky......... 13+9+3+1..........2,2,1,1
ESET...................15+12+1+2 .......2,2,1,1
Total Defense*...5+19+1+4.........1,3,1,2

Group 3
McAfee ............11+19+7+0.........2,3,2,1

Hard to classify
Trend Micro......0+0+82+175...... 1,1,4,4

The last column shows cluster analysis, which is most important. The number of missed samples is less important, but I have taken it as the secondary classification parameter (Avira, TotalDefense).
These tests cannot prove what is real protection of AVs in the same group (cluster analysis) - they can have the same protection in the wild despite a different number of missed samples.
Furthermore, the average detection difference between Group 1 and 2 is very little ( about 17 missed samples per 42000 total samples).
So, in practice, it would be hard to see the protection difference between most of the tested AVs (which is already a well known fact).

Malware Protection Test March 2019 - AV-Comparatives

The Malware Protection Test assesses a security program’s ability to protect a system against infection by malicious files before, during or after execution.

www.av-comparatives.org

Malware Protection Test September 2019 - AV-Comparatives

The Malware Protection Test assesses a security program’s ability to protect a system against infection by malicious files before, during or after execution.

www.av-comparatives.org

Malware Protection Test March 2020 - AV-Comparatives

The Malware Protection Test assesses a security program’s ability to protect a system against infection by malicious files before, during or after execution.

www.av-comparatives.org

Malware Protection Test September 2020 - AV-Comparatives

The Malware Protection Test assesses a security program’s ability to protect a system against infection by malicious files before, during or after execution.

www.av-comparatives.org

 

[plat1098]

Btw, he's not part of McAfee anymore. He sold it more than 20 years ago.

Yes, right, he still founded the company. I looked up the Wiki on McAfee; it's kind of amusing in spots. When Intel changed the name "McAfee" to Intel Security, he reportedly said: "I am now everlastingly grateful to Intel for freeing me from this terrible association with the worst software on the planet....."

What surprised me from that Wiki was that Intel still owns a 49% stake and Thoma Bravo, owner of Sophos, owns a minor stake also.

Source