AV-Comparatives Consumer Summary Report 2021

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Gandalf_The_Grey

Level 63
Thread author
Verified
Honorary Member
Top poster
Content Creator
Well-known
Apr 24, 2016
5,175
Austrian antivirus testing lab AV-Comparatives has released its annual Summary Report for consumer security products (PDF available here). This includes results of all the individual tests carried out during 2021, a user-interface review for all 17 tested products, and awards given to individual programs in various categories.

AV-Comparatives tested 17 Internet Security Products for Consumer

All tested products received the test lab’s Approved Security Product Award. This is given to all products that demonstrated they can provide competent protection against a wide variety of threats, without unduly affecting the computer’s performance or raising too many false alarms. The award is a certification of an effective and reliable product. The tested products were: Avast, AVG, Avira, Bitdefender, ESET, G Data, K7, Kaspersky, Malwarebytes, McAfee, Microsoft, NortonLifeLock, Panda, TotalAV, Total Defense, Trend Micro, and VIPRE.

AV-Comparatives awards McAfee with Product of the Year Award, Outstanding Product Awards go to Avast, AVG and Bitdefender, Top-Rated Product is Kaspersky.

Product of the Year


AV-Comparatives’ prestigious award goes to the program with the highest scores across all the test types. It is a demonstration of top quality in all categories. The 2021 Product of the Year Award goes to McAfee.

Outstanding Products

The Outstanding Product Award goes to any other product that reaches the same high award levels as the Product of the Year in the Main Test Series. This is proof of excellence across the board. Outstanding Product Awards for 2021 go to (alphabetically ordered) Avast, AVG and Bitdefender.

Top-Rated Product

Products that achieve high scores across all the test types are rewarded with Top-Rated Product Awards. These demonstrate that a program has achieved a high standard across all the test types. The 2021 Top-Rated Product Award goes to Kaspersky.

Bronze, Silver and Gold Awards for Individual Categories

For each individual test type, Bronze, Silver and Gold Awards go to the products that achieved the highest scores. The test types are:

Real-World Protection Test – this checks protection against Internet-borne threats
  • Gold: Bitdefender
  • Silver: Kaspersky, McAfee
  • Bronze: VIPRE
Malware Protection Test – this verifies a program’s ability to detect malicious files
  • Gold: McAfee
  • Silver: Bitdefender, VIPRE
  • Bronze: Total Defense
Performance Test – a check on how much a security product slows down the PC
  • Gold: Panda
  • Silver: Kaspersky
  • Bronze: ESET
Advanced Threat Protection Test – this optional test looks at protection against targeted attacks
  • Gold: Avast, AVG
  • Silver: ESET, Kaspersky, McAfee
  • Bronze: G Data
False-Positives Test – this assesses whether each program is prone to false alarms
  • Gold: ESET
  • Silver: Kaspersky
  • Bronze: TotalAV
As usual, the Summary Report includes a user-interface review of all the tested products. This provides a description of what each program is like to use in everyday scenarios. Topics covered include installation, security alerts, scan options, quarantine, logs, access control, and (where applicable) firewall.
Blog Post:
Summary Report:
PDF Download:
Twitter Post:
 
Last edited:

Mjolnir

Level 2
Verified
Jul 4, 2019
69
1
" Unfortunately, we cannot provide results or an award for Microsoft Defender in the February-May 2021 RealWorld Protection Test. During the testing, despite being configured for automatic updates and performing
manual updates, parts of Defender were not correctly updated. As there were no error messages, this issue was
only discovered at the beginning of June and required a new installation of the OS."
 

harlan4096

Moderator
Verified
Staff member
Malware Hunter
Well-known
Apr 28, 2015
8,061
McAfee actually improved a lot.
Product of the year! Seriously ???? 🙄

 

SeriousHoax

Level 43
Verified
Top poster
Well-known
Mar 16, 2019
3,171
Product of the year! Seriously ???? 🙄

It did well in AVC's test so what can we say 🤷‍♂️ McAfee didn't receive any special treatment. Good for them and congratulations on winning it.
Edit: I should add that I don't really like McAfee much.
 
Last edited:

Anthony Qian

Level 8
Verified
Well-known
Apr 17, 2021
357
Product of the year! Seriously ???? 🙄

Sample size matters. The results of the AV-C test are more reliable because it employs 10,000+ samples per test.

In addition, AV-C tests with Portable Executable files, whereas script malwares were used in Malware Hub tests.
 

silversurfer

Level 85
Verified
Honorary Member
Top poster
Content Creator
Malware Hunter
Well-known
Aug 17, 2014
7,788
Sample size matters. The results of the AV-C test are more reliable because it employs 10,000+ samples per test.

In addition, AV-C tests with Portable Executable files, whereas script malwares were used in Malware Hub tests.

You should better mention that even PE files are used on Malware Hub tests, that's a fact, just check VirusTotal links of Hub samples forums threads...

Just my two cents, Sample size doesn't matter for infected users, ONE sample only could be enough to infect a lot of users of any AV, that's what matters most in real life ;)
 

Anthony Qian

Level 8
Verified
Well-known
Apr 17, 2021
357
You should better mention that even PE files are used on Malware Hub tests, that's a fact, just check VirusTotal links of Hub samples forums threads...

Just my two cents, Sample size doesn't matter for infected users, ONE sample only could be enough to infect a lot of users of any AV, that's what matters most in real life ;)
Yes. I noted that Malware Hub's tests included both PE and script samples. However, in Malware Hub tests, McAfee mainly missed script malwares, which are not included in AV-C malware protection tests (but are included in ATP tests).

I would like to point out that no antivirus software can protect against all types of viruses. In AV-C malware tests, McAfee outperformed others, achieving a 100% detection rate. Kaspersky, which performs quite well in Malware Hub tests, missed 10 samples. So, based on the same theory, ONE missed sample could be enough to infect a lot of Kaspersky users and that's what matters most in real life.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top poster
Developer
Well-known
Dec 23, 2014
7,187
From what I could notice, Bitdefender scored well in this test 🙂 I say this because I see that certain products here in this forum, have an outcry from the community, while others are more “left aside”.

That can depend on what posts you read:


Bitdefender does not participate in all tests, so it is not included in the consumer results for the period 2019-2020.
If one takes the cumulative results of Bitdefender in the year 2021 (AV-test + AV-Comparatives consumer tests), then its protection is similar to the protection of a few top AVs.
 
Last edited:

Guilhermesene

Level 7
Verified
Well-known
Jun 1, 2019
332
Yes. I noted that Malware Hub's tests included both PE and script samples. However, in Malware Hub tests, McAfee mainly missed script malwares, which are not included in AV-C malware protection tests (but are included in ATP tests).

I would like to point out that no antivirus software can protect against all types of viruses. In AV-C malware tests, McAfee outperformed others, achieving a 100% detection rate. Kaspersky, which performs quite well in Malware Hub tests, missed 10 samples. So, based on the same theory, ONE missed sample could be enough to infect a lot of Kaspersky users and that's what matters most in real life.
I will make your words my own 🙂
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top poster
Developer
Well-known
Dec 23, 2014
7,187
...
I would like to point out that no antivirus software can protect against all types of viruses. In AV-C malware tests, McAfee outperformed others, achieving a 100% detection rate. Kaspersky, which performs quite well in Malware Hub tests, missed 10 samples. So, based on the same theory, ONE missed sample could be enough to infect a lot of Kaspersky users and that's what matters most in real life.

The malware tests are only a supplement to the more important (for home users) Real-World tests.
For example, Trend Micro missed over 200 samples in AV-Comparatives Malware Protection (consumer) tests in the year 2021 and Norton missed 0 samples. It does not mean that Norton is a much better AV for home users. In the same period of time, Trend Micro missed 0 samples in the AV-Comparatives Real-World tests.
 
Dec 12, 2021
230
The malware tests are only a supplement to the more important (for home users) Real-World tests.
For example, Trend Micro missed over 200 samples in AV-Comparatives Malware Protection (consumer) tests in the year 2021 and Norton missed 0 samples. It does not mean that Norton is a much better AV for home users. In the same period of time, Trend Micro missed 0 samples in the AV-Comparatives Real-World tests.
So independent labs tests are misleading, the average joe wont be able to understand that, all the tests to them is the same, AV vendors takes advantage of that by showing all these "awards", tricking users into thinking its actually good at detecting malware when it may aswell has just been web-based threats and may aswell be useless against actual malware executables.
 
Last edited:

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top poster
Developer
Well-known
Dec 23, 2014
7,187
AV-Comparatives Consumer Tests - cumulative results in the year 2021.

Real-World

------------------------ FEB-MAY-----JUL-OCT
Avast..................(1)................ 1
Avira................... 8.................. 6
Bitdefender.........1.................. 2
Eset.....................9.................. 8
Kaspersky...........4................... 1
Malwarebytes.....3................... 3
McAfee...............3.................. (3)
Microsoft.........*2.5................. 2
Norton..............2(3)................. 0
Panda................1.................... 0
TrendMicro.......0.................... 0
Vipre.................3.................... 2
Samples.........736................. 743

The number in parenthesis = user-dependent, they are counted as half missed.
* Microsoft was not tested in April-May, and in February-May 2021 parts of Defender were not correctly updated (The lab identified the issue in June). So the average for February-May has to be calculated on the basis of the known results from the period Feb 2019 to Mar 2021 (February-May 2021 excluded).


Malware Protection
-------------------- MAR------------SEP
Avast.................1............... 3
Avira................. 2............... 2
Bitdefender...... 0................ 2
Eset.................10.............. 18
Kaspersky....... 4................10
Malwarebytes. 6............... 16
McAfee............0............... 0
Microsoft.......15.............. 3
Norton........... 0................ 0
Panda............ 2................ 7
TrendMicro...103............121
Vipre.............. 0............... 2
Samples.....10013...... 10029
 
Last edited:

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top poster
Developer
Well-known
Dec 23, 2014
7,187
So independent labs tests are misleading, the average joe wont be able to understand that, all the tests to them is the same,

Probably yes.

AV vendors takes advantage of that by showing all these "awards", tricking users into thinking its actually good at detecting malware when it may aswell has just been web-based threats and may aswell be useless against actual malware executables.

The awards are OK - they include other important factors (not only protection). Furthermore, It is very probable that all of the AVs mentioned in my previous post can get similar protection (on default settings). The differences are too small to be important. If one uses Edge with SmartScreen and PUA enabled, then these differences may become meaningless.
 
Last edited:
Dec 12, 2021
230
Probably yes.



The awards are OK - they include other important factors (not only protection). Furthermore, It is very probable that all of the AVs mentioned in my previous post can get similar protection (on default settings). The differences are too small to be important. If one uses Edge with SmartScreen and PUA enabled, then these differences may become meaningless.
Still, the average user wont understand any of the awards it recieved, it provides people with an false sense of security given that these awards claim they have 100% (or near 100%) detection ratio, and yet we can see on the Malware Hubs results the weakness of AV's when it comes to scripts, indendent labs doesnt use malicious scripts not documents, which makes up probably more than half the worlds malware infections, especially with fileless malware, and the fact they give an PUP that just uses the Avira engine an 100% detection ratio.
 
Last edited: