AV-Comparatives Malware Protection Test March 2021

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

XLR8R

Level 4
Jan 20, 2020
160
723
Raiden said:
Most products today are very capable and will do a very good job in the real world. You know what will make the most improvement to your overall security compared to any AV tested?? You guessed it! Your overall computing hygiene. Poor habits lead to infection/problems eventually, it's just a matter of time...regardless of which AV you are using.

I am just a bit concerned about the decline compared to a year and two years ago for offline as well as online detection rates (not protection rate) for some products except the three which have kept their consistency.

It's not the question of how much one would have scored back in school, it's the question of whether you are able to keep yourself where you're at even if it's 75%. Some of the wildly varying scores of products like Trend Micro and Panda, or the gradually declining scores of Eset and Kaspersky (for example), are not awe-inspiring at all.
 

ChristianP

Level 1
May 1, 2020
35
103
All this test shows is that there are only three giants in AV engine and signatures/technology: BitDefender, Avast/AVG and Avira.

P.S. Eset and Kaspersky's slow decline is slightly concerning.
Kaspersky has shown from last year under 90% offline/online detection, it's weird, but it's protection it's still solid. Me, i'm happy with Norton for the last 12-14 months.
 

blackice

Level 33
Verified
Apr 1, 2019
2,215
13,008
I am just a bit concerned about the decline compared to a year and two years ago for offline as well as online detection rates (not protection rate) for some products except the three which have kept their consistency.

It's not the question of how much one would have scored back in school, it's the question of whether you are able to keep yourself where you're at even if it's 75%. Some of the wildly varying scores of products like Trend Micro and Panda, or the gradually declining scores of Eset and Kaspersky (for example), are not awe-inspiring at all.
I think it’s a reflection of their user bases moving to vectors of attack that are primarily online. Our household use is not at high risk when not online because all of our attack vectors would be from getting a file while online. I know there are different use cases where things like thumb drives and no internet come in to play, but this is probably what their telemetry is telling them. Most users’ infections come from online sources and they get infected while connected most of the time.
 

Nightwalker

Level 22
Verified
Trusted
Content Creator
May 26, 2014
1,189
7,901
I don't know why I still read the BS that is always posted in this kind of thread.

According to some users it may be better to ESET (outstanding product of 2020) and Kaspersky (product of the year 2020) just to leave the market, they are doomed, oh noes.
 

blackice

Level 33
Verified
Apr 1, 2019
2,215
13,008
I don't know why I still read the BS that is always posted in this kind of thread.

According to some users it may be better to ESET (outstanding product of 2020) and Kaspersky (product of the year 2020) just to leave the market, they are doomed, oh noes.
You’re right! They should all quit now that Malwarebytes is king! :ROFLMAO:

But seriously I have a license I’m going to give it a test soon.
 

Nightwalker

Level 22
Verified
Trusted
Content Creator
May 26, 2014
1,189
7,901
You’re right! They should all quit now that Malwarebytes is king! :ROFLMAO:

But seriously I have a license I’m going to give it a test soon.

Nobody can stop this Malware Samurai Crusher while its wearing Iron Man Armor: Mark VII with Captain American' s shield, malware industry is over, period.

02.jpg
 

Nightwalker

Level 22
Verified
Trusted
Content Creator
May 26, 2014
1,189
7,901
On a serious note, how much the difference between OFFLINE Detection Rate and ONLINE matters for the user?

Well, its something between zero and nothing, nada and nihil.

Some may say "ohh Nightwalker but but but what if there is no internet connection? What if malware destroy my precious source of porn delivery? I need "great" offline protection in that time hue hue".

Bad news for you, without constant updates your favorite offline great performer is useless too.


About Malwarebytes, for the real user, not for the pentester, it has always been pretty good, after all the product has a laser focus against PUPs, ransomwares (overrated btw) and Malvertising and those threats are a huge problem for most users, specially because some companies choose to ignore PUPs for legal reasons.
 
Last edited:

Raiden

Level 19
Verified
Content Creator
May 7, 2018
900
7,449
Bad news for you, without constant updates your favorite offline great performer is useless too.
Yup! A very solid point. Offline protection still need updates, so if you are offline over a significant period of time...your offline protection will suffer as time goes on.

It's not the question of how much one would have scored back in school, it's the question of whether you are able to keep yourself where you're at even if it's 75%. Some of the wildly varying scores of products like Trend Micro and Panda, or the gradually declining scores of Eset and Kaspersky (for example), are not awe-inspiring at all.

That is fair, but it still doesn't change the fact that the overall protection rate is still really high. I'm not saying to ignore offline, but online is much more of a realistic scenario in this day and age. Ignoring USBs, chances of getting infected while offline is extremely low regardless. Furthermore all of these products implore other methods to help with their overall protection. Regardless I think we are stressing about something that doesn't need to be stressed about. End of the day, the final results are still very good and I still think one is being sensational assuming that the likes of Kaspersky, Eset and others are going downhill because of one tiny aspect of the test, while ignoring everything else in the test. To be brutally honest, the fact that both Kaspersky and Eset were able to stop over 80% of the malware while being offline is still a pretty impressive showing. That's still over 8000 samples of the 10013 used in this test.(y):)
 

gonza

Level 1
Sep 10, 2019
26
131
IMHO, decisions should not be made based in just 1 factor.
Well, AVC say it better than me:

Please note that we do not recommend purchasing a product purely on the basis of one individual test or even one type of test. Rather, we would suggest that readers consult also our other recent test reports, and consider factors such as price, ease of use, compatibility and support. Installing a free trial version allows a program to be tested in everyday use before purchase.
 

Raiden

Level 19
Verified
Content Creator
May 7, 2018
900
7,449
I was going to say what I always say, that any of the major brands offer equivalent protection. Choose whatever works better for you. But I'm getting tired of repeating myself...
I won't lie I too am getting tired of sounding like a broken record. I feel that no matter how many times you say "there are multiple options to choose from, just use what works for you," many still just don't get it. There truly is no such thing a perfect security program. ALL security programs can miss malware, be bypassed, disabled, or whatever...hackers can be very determined when they want to be and no amount of 100% test results is ever going to change that.

It is in our nature as security geeks to obsess over these things at times...however, there comes a point where it does get a little sensational. I really wish that people would stop obsessing over tests, or a single aspect of a test and look at the bigger picture. Again I am going to sound like a broken record, but alas here it is...your overall security will largely be determined by your overall computing hygiene. Poor hygiene will lead to infection and problems, it's only a matter of time. Good hygiene will reduce your risk far greater than any security program will. Doesn't mean it still can't happen, but the chances will be significantly lower than those putting all of their hopes in a security program/setup and following poor hygiene.

If you just go around clicking/installing anything/everything, when your time does come and you loose all of your data because you didn't back it up, does it matter now that the AV you were using got 96% on an offline test? No it doesn't because your data is gone and it's not coming back, but don't worry it got 96% offline so how is this possible?🤯

IMHO, decisions should not be made based in just 1 factor.
Well, AVC say it better than me:
How about that, even the testing organization is saying to take these tests with a grain of salt, look at the bigger picture and to essentially say that no product if perfect, use the one that best suites your needs.:unsure:
 
Last edited:

XLR8R

Level 4
Jan 20, 2020
160
723
I won't lie I too am getting tired of sounding like a broken record. I feel that no matter how many times you say "there are multiple options to choose from, just use what works for you," many still just don't get it. There truly is no such thing a perfect security program. ALL security programs can miss malware, be bypassed, disabled, or whatever...hackers can be very determined when they want to be and no amount of 100% test results is ever going to change that.

It is in our nature as security geeks to obsess over these things at times...however, there comes a point where it does get a little sensational. I really wish that people would stop obsessing over tests, or a single aspect of a test and look at the bigger picture. Again I am going to sound like a broken record, but alas here it is...your overall security will largely be determined by your overall computing hygiene. Poor hygiene will lead to infection and problems, it's only a matter of time. Good hygiene will reduce your risk far greater than any security program will. Doesn't mean it still can't happen, but the chances will be significantly lower than those putting all of their hopes in a security program/setup and following poor hygiene.

If you just go around clicking/installing anything/everything, when your time does come and you loose all of your data because you didn't back it up, does it matter now that the AV you were using got 96% on an offline test? No it doesn't because your data is gone and it's not coming back, but don't worry it got 96% offline so how is this possible?🤯


How about that, even the testing organization is saying to take these tests with a grain of salt, look at the bigger picture and to essentially say that no product if perfect, use the one that best suites your needs.:unsure:

To be really honest, I was always surprised at the FP test results since in real use I never faced those FPs with for e.g. Norton or K7. But after speaking to a few vendors I found that FP rates tend to be higher for the products that do not have a strong presence in the geographical area of the concerned tester and this is related to the lower userbase of the vendors in those regions that they are not able (or not inclined) to screen/whitelist properly the localized software/files popular there (take for e.g. Norton vs. Avira in the FP test in AV-C). It's a bit weird but I think your geographic region matters a bit when you are selecting an anti-malware product....
 
Last edited:

Andy Ful

Level 73
Verified
Trusted
Developer
Dec 23, 2014
6,284
42,892
Provided, the internet connection works just fine..... :)

Though, real-world protection test may be more representative, considering behavioral protection layers as well. But I still think the scan engine itself matters (technology + sigs + heur + cloud).
Of course. If most AVs were dependent only on the cloud, the malc0ders would develop attacks that could exploit it. TrendMicro protection can be effective, because most AVs still have very good signatures even for threats that have never happen in the wild on the computers protected by a particular AV.
 

Cortex

Level 26
Verified
Aug 4, 2016
1,499
8,288
The graph now starts at 95% which for example makes Trend & MS for example look far worse than the small increment of compromised than it is - I've noticed comparatives seem to keep moving the percentage the graph starts up higher ?
 

Freki123

Level 9
Verified
Aug 10, 2013
405
1,437
To be really honest, I was always surprised at the FP test results since in real use I never faced those FPs with for e.g. Norton or K7.
A week ago I tested K7 to find out if I would like it. After getting 2 fp in under 15 mins of using it I got bored and stopped my test. I wasn't looking for a fp just downloading an legit installer and running another clean software on my pc (European country here).
 

Divine_Barakah

Level 27
Verified
May 10, 2019
1,617
8,258
A week ago I tested K7 to find out if I would like it. After getting 2 fp in under 15 mins of using it I got bored and stopped my test. I wasn't looking for a fp just downloading an legit installer and running another clean software on my pc (European country here).
Been using K7 for months and have not encountered any FPs. I do not run any unusual nor uncommon programmes.
 

Nagisa

Level 7
Verified
Jul 19, 2018
340
1,452
An unpopular opinion maybe but I don't think those protection rates are realistic. The easiest way to see this is from looking at hub tests or manually downloading packs of malware from places like abuse.ch and throwing them to AVs.
 

Nightwalker

Level 22
Verified
Trusted
Content Creator
May 26, 2014
1,189
7,901
An unpopular opinion maybe but I don't think those protection rates are realistic. The easiest way to see this is from looking at hub tests or manually downloading packs of malware from places like abuse.ch and throwing them to AVs.

It is true, fair enough, but it is not realistic either to download packs of malware and execute them.
 

Raiden

Level 19
Verified
Content Creator
May 7, 2018
900
7,449
To be really honest, I was always surprised at the FP test results since in real use I never faced those FPs with for e.g. Norton or K7. But after speaking to a few vendors I found that FP rates tend to be higher for the products that do not have a strong presence in the geographical area of the concerned tester and this is related to the lower userbase of the vendors in those regions that they are not able (or not inclined) to screen/whitelist properly the localized software/files popular there (take for e.g. Norton vs. Avira in the FP test in AV-C). It's a bit weird but I think your geographic region matters a bit when you are selecting an anti-malware product....

It is true and makes sense!

If a certain AV is highly used in the USA, but used very little Europe for example, it's FP rate and detection rate of certain malware families will be different than an AV highly used in Europe vs USA. Some pieces of software may only be available/used in certain regions. Similarly some malware families are only seen in certain regions as well. With the advent of ML and the cloud I would assume that FPs are going to be higher generally speaking, as most AVs seem to be imporing some sort of file reputation system. MD and Smartscreen are a great example of this. Between BAFS and Smartscreen, MD/W10 can be pretty ruthless to new files. To the point that it can be extremely annoying for a developer trying to create a new program.

The problem with FPs in tests (at least the way AV-Comparatives displays them) is you only see the total number as part of the graph/chart. AV-Comparatives use to post an appendix to outline in more detail where the FPs came from, as well as if the file had very low, low, medium, or high prevalence. Using MD as an example, it use to have reasonably high FPs in AV-Comparatives. However, when looking at the appendix, one would see that the bulk of the FPs came from files with very low to low prevalence. Files with medium, or high prevalence were extremely low. Which agrees with what the vendors are telling you. Most people in general use software that is fairly well known to most AV vendors, so the FP rate in the real world for the vast majority of people is going to be really low. However, if you are using an AV that is not widely used in your region and you are either using software that is primarily used in that region, or a file with low prevalence in general, your chances of a FP will be higher.

It is true, fair enough, but it is not realistic either to download packs of malware and execute them.
But..but, all those YouTuber tests always showed me that a large folder of random files will just randomly appear on my desktop one day. They also showed me that it's best practice to just execute each one to see what it is, even though it just randomly appeared....:p
 
Top