AV-Comparatives Consumer Summary Report 2021

[Gandalf_The_Grey]

Austrian antivirus testing lab AV-Comparatives has released its annual Summary Report for consumer security products (PDF available here). This includes results of all the individual tests carried out during 2021, a user-interface review for all 17 tested products, and awards given to individual programs in various categories.

AV-Comparatives tested 17 Internet Security Products for Consumer

All tested products received the test lab’s Approved Security Product Award. This is given to all products that demonstrated they can provide competent protection against a wide variety of threats, without unduly affecting the computer’s performance or raising too many false alarms. The award is a certification of an effective and reliable product. The tested products were: Avast, AVG, Avira, Bitdefender, ESET, G Data, K7, Kaspersky, Malwarebytes, McAfee, Microsoft, NortonLifeLock, Panda, TotalAV, Total Defense, Trend Micro, and VIPRE.

AV-Comparatives awards McAfee with Product of the Year Award, Outstanding Product Awards go to Avast, AVG and Bitdefender, Top-Rated Product is Kaspersky.

Product of the Year

AV-Comparatives’ prestigious award goes to the program with the highest scores across all the test types. It is a demonstration of top quality in all categories. The 2021 Product of the Year Award goes to McAfee.

Outstanding Products

The Outstanding Product Award goes to any other product that reaches the same high award levels as the Product of the Year in the Main Test Series. This is proof of excellence across the board. Outstanding Product Awards for 2021 go to (alphabetically ordered) Avast, AVG and Bitdefender.

Top-Rated Product

Products that achieve high scores across all the test types are rewarded with Top-Rated Product Awards. These demonstrate that a program has achieved a high standard across all the test types. The 2021 Top-Rated Product Award goes to Kaspersky.

Bronze, Silver and Gold Awards for Individual Categories

For each individual test type, Bronze, Silver and Gold Awards go to the products that achieved the highest scores. The test types are:

Real-World Protection Test – this checks protection against Internet-borne threats

• Gold: Bitdefender
• Silver: Kaspersky, McAfee
• Bronze: VIPRE

Malware Protection Test – this verifies a program’s ability to detect malicious files

• Gold: McAfee
• Silver: Bitdefender, VIPRE
• Bronze: Total Defense

Performance Test – a check on how much a security product slows down the PC

• Gold: Panda
• Silver: Kaspersky
• Bronze: ESET

Advanced Threat Protection Test – this optional test looks at protection against targeted attacks

• Gold: Avast, AVG
• Silver: ESET, Kaspersky, McAfee
• Bronze: G Data

False-Positives Test – this assesses whether each program is prone to false alarms

• Gold: ESET
• Silver: Kaspersky
• Bronze: TotalAV

As usual, the Summary Report includes a user-interface review of all the tested products. This provides a description of what each program is like to use in everyday scenarios. Topics covered include installation, security alerts, scan options, quarantine, logs, access control, and (where applicable) firewall.

Blog Post:

Consumer Summary Report 2021

Austrian antivirus testing lab AV-Comparatives has released its annual Summary Report for consumer security products (PDF available here). This includes

www.av-comparatives.org

Summary Report:

Summary Report 2021

Read the Summary Report 2021 to learn more about the various AV products tested over the year and the high-scoring products in these tests.

www.av-comparatives.org

PDF Download:

https://www.av-comparatives.org/wp-content/uploads/2022/01/avc_sum_202112.pdf

Twitter Post:

 

[Anthony Qian]

McAfee actually improved a lot.

 

[Mjolnir]

1
" Unfortunately, we cannot provide results or an award for Microsoft Defender in the February-May 2021 RealWorld Protection Test. During the testing, despite being configured for automatic updates and performing
manual updates, parts of Defender were not correctly updated. As there were no error messages, this issue was
only discovered at the beginning of June and required a new installation of the OS."

 

[harlan4096]

McAfee actually improved a lot.

Product of the year! Seriously ????

McAfee TP IS - January 2022 Report

McAfee TP - January 2022 Report Due to the small number of samples used in this tests, you should take results with a grain of salt. We encourage you to compare these results with others and take informed decisions on what security products to use. __ System Status Abbreviations: P ...

malwaretips.com

 

[SeriousHoax]

Product of the year! Seriously ????

McAfee TP IS - January 2022 Report

McAfee TP - January 2022 Report Due to the small number of samples used in this tests, you should take results with a grain of salt. We encourage you to compare these results with others and take informed decisions on what security products to use. __ System Status Abbreviations: P ...

malwaretips.com

It did well in AVC's test so what can we say McAfee didn't receive any special treatment. Good for them and congratulations on winning it.
Edit: I should add that I don't really like McAfee much.

 

[Anthony Qian]

Product of the year! Seriously ????

McAfee TP IS - January 2022 Report

McAfee TP - January 2022 Report Due to the small number of samples used in this tests, you should take results with a grain of salt. We encourage you to compare these results with others and take informed decisions on what security products to use. __ System Status Abbreviations: P ...

malwaretips.com

Sample size matters. The results of the AV-C test are more reliable because it employs 10,000+ samples per test.

In addition, AV-C tests with Portable Executable files, whereas script malwares were used in Malware Hub tests.

 

[ScandinavianFish]

As if these tests were any less believable they just had to put mcafee at the top.

 

[harlan4096]

McAfee is a disaster with malware different from exe files...

 

[Nightwalker]

McAfee is a disaster with malware different from exe files...

McAfee is a disaster

 

[silversurfer]

Sample size matters. The results of the AV-C test are more reliable because it employs 10,000+ samples per test.

In addition, AV-C tests with Portable Executable files, whereas script malwares were used in Malware Hub tests.

You should better mention that even PE files are used on Malware Hub tests, that's a fact, just check VirusTotal links of Hub samples forums threads...

Just my two cents, Sample size doesn't matter for infected users, ONE sample only could be enough to infect a lot of users of any AV, that's what matters most in real life

 

[Guilhermesene]

From what I could notice, Bitdefender scored well in this test I say this because I see that certain products here in this forum, have an outcry from the community, while others are more “left aside”.

 

[Anthony Qian]

You should better mention that even PE files are used on Malware Hub tests, that's a fact, just check VirusTotal links of Hub samples forums threads...

Just my two cents, Sample size doesn't matter for infected users, ONE sample only could be enough to infect a lot of users of any AV, that's what matters most in real life

Yes. I noted that Malware Hub's tests included both PE and script samples. However, in Malware Hub tests, McAfee mainly missed script malwares, which are not included in AV-C malware protection tests (but are included in ATP tests).

I would like to point out that no antivirus software can protect against all types of viruses. In AV-C malware tests, McAfee outperformed others, achieving a 100% detection rate. Kaspersky, which performs quite well in Malware Hub tests, missed 10 samples. So, based on the same theory, ONE missed sample could be enough to infect a lot of Kaspersky users and that's what matters most in real life.

 

[Andy Ful]

From what I could notice, Bitdefender scored well in this test I say this because I see that certain products here in this forum, have an outcry from the community, while others are more “left aside”.

That can depend on what posts you read:

MRG Effitas - MRG Effitas 360 Degree Assessment & Certification Q2 2021

https://www.mrg-effitas.com/wp-content/uploads/2021/08/MRG_Effitas_360_Q2_2021.pdf This test is for businesses, so the tested AVs have some Advanced Threat Protection features. The results should not be compared to the results of AV Home versions. Non-default endpoint protection...

malwaretips.com

The best Home AV protection 2019-2020

Real-World tests include fresh web-originated samples. Malware Protection tests include older samples (several days old) usually delivered via USB drives or network drives. Real-World 2019-2020: SE Labs, AV-Comparatives, AV-Test (7659 samples in 24 tests) -------------------Missed samples...

malwaretips.com

Bitdefender does not participate in all tests, so it is not included in the consumer results for the period 2019-2020.
If one takes the cumulative results of Bitdefender in the year 2021 (AV-test + AV-Comparatives consumer tests), then its protection is similar to the protection of a few top AVs.

 

[Guilhermesene]

Yes. I noted that Malware Hub's tests included both PE and script samples. However, in Malware Hub tests, McAfee mainly missed script malwares, which are not included in AV-C malware protection tests (but are included in ATP tests).

I would like to point out that no antivirus software can protect against all types of viruses. In AV-C malware tests, McAfee outperformed others, achieving a 100% detection rate. Kaspersky, which performs quite well in Malware Hub tests, missed 10 samples. So, based on the same theory, ONE missed sample could be enough to infect a lot of Kaspersky users and that's what matters most in real life.

I will make your words my own

 

[Andy Ful]

...
I would like to point out that no antivirus software can protect against all types of viruses. In AV-C malware tests, McAfee outperformed others, achieving a 100% detection rate. Kaspersky, which performs quite well in Malware Hub tests, missed 10 samples. So, based on the same theory, ONE missed sample could be enough to infect a lot of Kaspersky users and that's what matters most in real life.

The malware tests are only a supplement to the more important (for home users) Real-World tests.
For example, Trend Micro missed over 200 samples in AV-Comparatives Malware Protection (consumer) tests in the year 2021 and Norton missed 0 samples. It does not mean that Norton is a much better AV for home users. In the same period of time, Trend Micro missed 0 samples in the AV-Comparatives Real-World tests.

 

[ScandinavianFish]

The malware tests are only a supplement to the more important (for home users) Real-World tests.
For example, Trend Micro missed over 200 samples in AV-Comparatives Malware Protection (consumer) tests in the year 2021 and Norton missed 0 samples. It does not mean that Norton is a much better AV for home users. In the same period of time, Trend Micro missed 0 samples in the AV-Comparatives Real-World tests.

So independent labs tests are misleading, the average joe wont be able to understand that, all the tests to them is the same, AV vendors takes advantage of that by showing all these "awards", tricking users into thinking its actually good at detecting malware when it may aswell has just been web-based threats and may aswell be useless against actual malware executables.

 

[Andy Ful]

AV-Comparatives Consumer Tests - cumulative results in the year 2021.

Real-World
------------------------ FEB-MAY-----JUL-OCT
Avast..................(1)................ 1
Avira................... 8.................. 6
Bitdefender.........1.................. 2
Eset.....................9.................. 8
Kaspersky...........4................... 1
Malwarebytes.....3................... 3
McAfee...............3.................. (3)
Microsoft.........*2.5................. 2
Norton..............2(3)................. 0
Panda................1.................... 0
TrendMicro.......0.................... 0
Vipre.................3.................... 2
Samples.........736................. 743

The number in parenthesis = user-dependent, they are counted as half missed.
* Microsoft was not tested in April-May, and in February-May 2021 parts of Defender were not correctly updated (The lab identified the issue in June). So the average for February-May has to be calculated on the basis of the known results from the period Feb 2019 to Mar 2021 (February-May 2021 excluded).


Malware Protection
-------------------- MAR------------SEP
Avast.................1............... 3
Avira................. 2............... 2
Bitdefender...... 0................ 2
Eset.................10.............. 18
Kaspersky....... 4................10
Malwarebytes. 6............... 16
McAfee............0............... 0
Microsoft.......15.............. 3
Norton........... 0................ 0
Panda............ 2................ 7
TrendMicro...103............121
Vipre.............. 0............... 2
Samples.....10013...... 10029

 

[RoboMan]

Sample size matters. The results of the AV-C test are more reliable because it employs 10,000+ samples per test.

??????????????????????

 

[Andy Ful]

So independent labs tests are misleading, the average joe wont be able to understand that, all the tests to them is the same,

Probably yes.

AV vendors takes advantage of that by showing all these "awards", tricking users into thinking its actually good at detecting malware when it may aswell has just been web-based threats and may aswell be useless against actual malware executables.

The awards are OK - they include other important factors (not only protection). Furthermore, It is very probable that all of the AVs mentioned in my previous post can get similar protection (on default settings). The differences are too small to be important. If one uses Edge with SmartScreen and PUA enabled, then these differences may become meaningless.

 

[ScandinavianFish]

Probably yes.



The awards are OK - they include other important factors (not only protection). Furthermore, It is very probable that all of the AVs mentioned in my previous post can get similar protection (on default settings). The differences are too small to be important. If one uses Edge with SmartScreen and PUA enabled, then these differences may become meaningless.

Still, the average user wont understand any of the awards it recieved, it provides people with an false sense of security given that these awards claim they have 100% (or near 100%) detection ratio, and yet we can see on the Malware Hubs results the weakness of AV's when it comes to scripts, indendent labs doesnt use malicious scripts not documents, which makes up probably more than half the worlds malware infections, especially with fileless malware, and the fact they give an PUP that just uses the Avira engine an 100% detection ratio.