AV-Comparatives Consumer Summary Report 2021

[Gandalf_The_Grey]

From AV-Comparatives:

Advice on Choosing Computer Security Software

There is no such thing as the perfect security program, or the best one for all needs and every user. Being recognized as “Product of the Year” does not mean that a program is the “best” in all cases and for everyone: it only means that its overall performance in our tests throughout the year was consistent and unbeaten. Before selecting a security product, please visit the vendor’s website and evaluate their software by downloading a trial version. Our awards are based on test results only and do not consider other important factors (such as available interface languages, price, and support options), which you should evaluate for yourself.

Further info in the report:
Pricing and auto-renewal

In the table below we have listed the (rounded) current discount price, full list price and auto-renewal prices (where applicable), including sales tax, for the paid products in the 2021 Main Test Series. We note that nearly all vendors show prices for consumer products on their websites with taxes included, but this is not guaranteed. You might like to check on this before buying a product. Please also note that changes in currency exchange rates may influence the price in some cases.

Help and support for technical issues

Sorry, AV-Comparatives does not provide technical support for any product. However, if you need assistance with your AV product, we have listed below some of the English-language help and support options for the products in our Consumer Main Test Series. You can click on the links to go directly to the relevant pages of the respective products’ websites.

Product	Online Help	Support Forum	Contact Support
Avast Free Antivirus	Online Help	Avast Forum	n/a
AVG Antivirus Free	Online Help	AVG Forum	n/a
Avira Antivirus Pro	Online Help	Avira Forum	Contact
Bitdefender Internet Security	Online Help	Bitdefender Forum	Contact
ESET Internet Security	Online Help	ESET Forum	Contact
G Data Total Security	Online Help	n/a	Contact
K7 Total Security	Online Help	K7 Forum	Contact
Kaspersky Internet Security	Online Help	Kaspersky Forum	Contact
Malwarebytes Premium	Online Help	Malwarebytes Forum	Contact
McAfee Total Protection	Online Help	McAfee Forum	Contact
Microsoft Defender Antivirus	Online Help	Microsoft Forum	n/a
NortonLifeLock Norton 360	Online Help	NortonLifeLock Forum	Contact
Panda Free Antivirus	Online Help	Panda Forum	n/a
TotalAV Total Security	Online Help	n/a	Contact
Total Defense Essential Antivirus	Online Help	n/a	Contact
Trend Micro Internet Security	Online Help	Trend Micro Forum	Contact
VIPRE Advanced Security	Online Help	VIPRE Forum	Contact

Product Reviews

The aim of the user-experience review is to give readers an idea of what each tested product is like to use in everyday situations.

Despite all the criticism, I think this is the best and most comprehensive report on antivirus software out there.

 

[Zorro]

The first thing I did when I bought a laptop is to remove the pre-installed McAfee antivirus. Because his license has long expired, but I did not want to pay for it. And I would give the gold medal to this antivirus in the nomination "The most difficult uninstalled product of the year." At first I deleted it using a standard system uninstaller. But I decided that you still need to try to use the special utility to remove the McAfee. I launched this utility and she removed something long there. After rebooting, I decided that everything is in order. I manually checked the folders (including hidden), in which the remnants of McAfee could remain, and removed several such residues. But after a few months, using the Autoruns program, I saw that in the driver folder there are still 5 McAfee drivers!!! I deleted these drivers and decided to manually check the registry of Windows. And when I manually checked the registry, I found another about 10-15 entries belonging to McAfee. I am still not sure that McAfee did not hide somewhere in the depths of Windows.

 

[superleeds27]

Product of the year! Seriously ????

McAfee TP IS - January 2022 Report

McAfee TP - January 2022 Report Due to the small number of samples used in this tests, you should take results with a grain of salt. We encourage you to compare these results with others and take informed decisions on what security products to use. __ System Status Abbreviations: P ...

malwaretips.com

Haha, Noticed an increase of spam/scam/phishing emails for McAfee too lately. 'My subscripton is ending sooon' - 'license is ending' - 'licence needs renewed'

Sigh....

 

[Zorro]

In the performance test strange results. I used Kaspersky Internet Security a year and a half ago, and the last year and a half I use Eset Inerture Securiti. And I can say that Eset is much "easier" than Kaspersky, and the system has become significantly faster to work with Eset than with Kaspersky. So at least Eset and Kaspersky should be swapped in these results. I will not say about Antivirus Panda, because I did not use, but I read on the forums, that this is no longer such a "fluff", which was before.

 

[superleeds27]

In the performance test strange results. I used Kaspersky Internet Security a year and a half ago, and the last year and a half I use Eset Inerture Securiti. And I can say that Eset is much "easier" than Kaspersky, and the system has become significantly faster to work with Eset than with Kaspersky. So at least Eset and Kaspersky should be swapped in these results. I will not say about Antivirus Panda, because I did not use, but I read on the forums, that this is no longer such a "fluff", which was before.

I was just looking at that section too. I thought it would have been the other way around?

 

[ScandinavianFish]

From AV-Comparatives:

Further info in the report:
Pricing and auto-renewal

Help and support for technical issues

Product Reviews

Despite all the criticism, I think this is the best and most comprehensive report on antivirus software out there.

So they have one 3 devices and one 6 devices, then compare them to the price of one device for the rest of the vendors? that makes no sense.

 

[Gandalf_The_Grey]

So they have one 3 devices and one 6 devices, then compare them to the price of one device for the rest of the vendors? that makes no sense.

If that is the cheapest official option, it makes sense.

But most of the time there much better offers can be found when you don't shop on the official website.

What worries me is that so many have mandatory auto-renewal.
That way people that don't know any better pay way too much for their av subscription.

 

[Andy Ful]

Still, the average user wont understand any of the awards it recieved, it provides people with an false sense of security given that these awards claim they have 100% (or near 100%) detection ratio, and yet we can see on the Malware Hubs results the weakness of AV's when it comes to scripts,

We may skip average users here. They do not care about AV tests. They will use any AV suggested by someone who is accepted by them as a more experienced user.
These tests do not provide people with a false sense of security, because the awarded AVs are as good (in practice) as the best AVs tested on Malware Hub. This is not in contradiction with MH results.

The MH samples include both the samples used as initial infection vectors and also many samples that are payloads (delivered later in the infection chain). Such samples are relevant in Enterprises - payloads can be directly used to infect (clean) computers in the network via lateral movement. Furthermore, many samples on MH come from targeted attacks (rarely used against home users).
The Real-World AV-Comparatives Consumer tests are focused on the samples that can be initial vectors of infection against home users, because there is no lateral movement in the home environment. The payloads and malware from targeted attacks are not common in these tests.

So MH and Consumers Real-World tests are relevant for different environments and the results cannot be directly compared.

indendent labs doesnt use malicious scripts not documents, which makes up probably more than half the worlds malware infections, especially with fileless malware, ...

The Real-World tests include malicious URLs and executable files that were downloaded from these URLs. There are several definitions of executable files. For example:

During inventory, the application detects executable files of the following formats: MZ, COM, PE, NE, SYS, CMD, BAT, PS1, JS, VBS, REG, MSI, CPL, DLL, JAR, and HTML files.

Inventory of executable files

support.kaspersky.com

On Fileinfo.com we can see near 200 such executable files:

Executable File Formats

A list of executable file extensions and their associated file formats.

fileinfo.com

It is not clear which files are treated by AV-Comparatives as executable files, but it is very probable that this can be any file that can be downloaded/run via malicious URL (also scripts).

 

[ScandinavianFish]

We may skip average users here. They do not care about AV tests. They will use any AV suggested by someone who is accepted by them as a more experienced user.
These tests do not provide people with a false sense of security, because the awarded AVs are as good (in practice) as the best AVs tested on Malware Hub. This is not in contradiction with MH results.

The MH samples include both the samples used as initial infection vectors and also many samples that are payloads (delivered later in the infection chain). Such samples are relevant in Enterprises - payloads can be directly used to infect (clean) computers in the network via lateral movement. Furthermore, many samples on MH come from targeted attacks (rarely used against home users).
The Real-World AV-Comparatives Consumer tests are focused on the samples that can be initial vectors of infection against home users, because there is no lateral movement in the home environment. The payloads and malware from targeted attacks are not common in these tests.

So MH and Consumers Real-World tests are relevant for different environments and the results cannot be directly compared.



The Real-World tests include malicious URLs and executable files that were downloaded from these URLs. There are several definitions of executable files. For example:



On Fileinfo.com we can see near 200 such executable files:

Executable File Formats

A list of executable file extensions and their associated file formats.

fileinfo.com

It is not clear which files are treated by AV-Comparatives as executable files, but it is very probable that this can be any file that can be downloaded/run via malicious URL (also scripts).

Still, such files definitely make up an small percentage of the actual executable files, which means, according to the malware hub, they would have had some misses, yet a lot of these products consistently score 100% or near 100% against malware.

 

[Andy Ful]

Still, such files definitely make up an small percentage of the actual executable files, which means, according to the malware hub, they would have had some misses,

You have this problem to solve:
Which AV is better? The one that is minimally better on common threats, or another one that is significantly better on very rare threats.

The current testing methods cannot precisely measure in one particular test which AV is the best for common threats. Also, the very rare threats for home users can be common in Enterprises. That is why you cannot rely here on MH results in the context of home AVs for home users.

yet a lot of these products consistently score 100% or near 100% against malware.

Still, the significant differences can be seen in cumulative results.
https://malwaretips.com/threads/the-best-home-av-protection-2019-2020.106485/post-927440

Currently, it is not possible to perform the one reliable test that could show the real difference between several popular AVs. Such tests have to be done constantly for a long time. Only the cumulative results make sense.
This is true for all tests, including MH tests.

Some time ago I prepared the cumulative results for Kaspersky on the basis of MH tests. Here is my conclusion:

When comparing it to the previous result we can see how unreliable can be taking a few hundreds of malware samples. The first 214 samples gave 1 infection and the next 335 gave 6. Anyway, the extended analysis suggests that Malware Hub samples are fresher as compared to those used by AV-Comparatives.

https://malwaretips.com/threads/randomness-in-the-av-labs-testing.104104/post-905706

Edit.
I would like to make a correction to my old conclusion. The greater number of missed samples in MH tests compared to AV testing Labs Consumer tests, can follow from the used samples. There can be much more samples from targeted attacks in MH tests, compared to Consumer tests.

 

[oldschool]

You have this problem to solve:
Which AV is better? The one that is minimally better on common threats, or another one that is significantly better on very rare threats.

The current testing methods cannot precisely measure in one particular test which AV is the best for common threats. Also, the very rare threats for home users can be common in Enterprises. That is why you cannot rely here on MH results in the context of home AVs for home users.

Still, the significant differences can be seen in cumulative results.
The best Home AV protection 2019-2020

Currently, it is not possible to perform the one reliable test that could show the real difference between several popular AVs. Such tests have to be done constantly for a long time. Only the cumulative results make sense.
This is true for all tests, including MH tests.

Some time ago I prepared the cumulative results for Kaspersky on the basis of MH tests. Here is my conclusion:


Edit.
I would like to make a correction to my old conclusion. The greater number of missed samples in MH tests compared to AV testing Labs Consumer tests, can follow from the used samples. There can be much more samples from targeted attacks in MH tests, compared to Consumer tests.

You might as well bookmark this answer (or make a Word template) which you have used often, for good reason, in threads such as these. It will save you time posting in the future!

Currently, it is not possible to perform the one reliable test that could show the real difference between several popular AVs. Such tests have to be done constantly for a long time. Only the cumulative results make sense.
This is true for all tests, including MH tests.

This is the essential truth, and apparently it needs to be said repeatedly in this kind of thread because it apparently doesn't sink in for some readers.

 

[Andy Ful]

The problem of reliable AV testing is very complex and requires a lot of resources and time. The current testing methodologies were developed for over 20 years and they are still far from being perfect. Each of the well-respected AV testing labs uses a slightly different methodology, so one has to be careful when comparing their results.

The main problem for the readers is understanding that the results of one particular test cannot reflect the real protection, because the tested samples are only a small percent of all samples in the wild. Furthermore, different AVs can miss different samples and detect some samples missed by other AVs.

Usually, the chance that a particular AV can miss 0 samples in one particular test is similar to missing a few samples in another test, even if the samples were chosen from the same set of all samples available in the wild. This can be calculated and follows from statistical considerations. One has to use statistics because there is a tremendous number of ways to choose a small pule of samples (usually a few hundred) from all samples available in the wild (usually a few million). Each choice can give a slightly different number of missed samples and has nothing to do with a lower or higher AV protection.

So, if the reader can see (in the AV-Comparatives Real-World tests) that Kaspersky missed 0 samples in March and 3 samples in September, this does not prove in any way that the protection of Kaspersky was lower in September. We can see this constantly in the tests for most AVs.
Similarly, if Kaspersky missed 3 samples in a particular test and another AV missed 0 samples in the same test, this does not prove in any way that the protection of Kaspersky was lower compared to this AV. This uncertainty is caused by using a random pule of tested samples, because this pule of samples is a small part of all available samples (all samples in the world).

 

[ExecutiveOrder]

From what I could notice, Bitdefender scored well in this test I say this because I see that certain products here in this forum, have an outcry from the community, while others are more “left aside”.

If only Bitdefender could perform better in ATP test, "Outstanding Products" won't exist this year and there will be only sole and pure "Product of the Year", also Kaspersky will be kicked out from "Top-rated Product".

 

[Andy Ful]

the definition of insanity is doing the same thing over and over, but expecting a different result
dissecting av test lab reports is just like that

View attachment 263690

Ha, ha.
The AV testing is slightly different. The AV testing labs are doing the same thing over and over, but know that slightly different results between tested AVs are not related to different protection.
The problem is that this knowledge is not common among other people.

 

[RoboMan]

Umbra Total Security would have nailed these tests. If only it hadn't been disqualified for being too powerful...

 

[Andy Ful]

Umbra Total Security would have nailed these tests. If only it hadn't been disqualified for being too powerful...

Umbra Total Security Test and Review

Due to the lack of efficacy of currently available anti-malware applications it was determined that an extensive unbiased test should be run on the most excellent newcomer to the security market, Umbra Total Security. Materials and Methods- A malware sample set consisting of 143,213 unique...

malwaretips.com

Smile and enjoy this thread. Nothing could win UTS.

 

[Andy Ful]

... it is about the transparency and education of the consumer, ...

Some consumers can benefit from these tests and some not.

 

[Andy Ful]

those consumers that actually bother to read them and can understand them

Yes. For most readers, the test reports are not especially useful. But currently, the AV testing labs represent the most objective platform for AV vendors. They pay a lot of money for testing and the popular vendors do not complain much, even when getting horrible results sometimes.

Edit.
Furthermore, such tests prevent some vendors from insisting that their products are the best, when they are only average.