AV-Comparatives Malware Protection Test March 2021

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Nagisa

Level 7
Verified
Jul 19, 2018
341
but it is not realistic either to download packs of malware and execute them.
That's the same as dismissing all AV tests because all they are doing the same (executing thousands of malware) after all. I think this argument, in general*, is a bit like dismissing car crash tests as you won't intentionally drive to a wall with high speed in the real world.

*:
But..but, all those YouTuber tests always showed me that a large folder of random files will just randomly appear on my desktop one day. They also showed me that it's best practice to just execute each one to see what it is, even though it just randomly appeared....:p


The easiness of finding big number of malicious files on internet that gets undetected by my AV makes me think why should I even bother using it. That's making the big majority of AVs just a tool to keep average Joe from executing not-a-malware.exe
 
  • Like
Reactions: Nevi and Cortex

mlnevese

Level 26
Verified
Top Poster
Well-known
May 3, 2015
1,531
It is true and makes sense!

If a certain AV is highly used in the USA, but used very little Europe for example, it's FP rate and detection rate of certain malware families will be different than an AV highly used in Europe vs USA. Some pieces of software may only be available/used in certain regions. Similarly some malware families are only seen in certain regions as well. With the advent of ML and the cloud I would assume that FPs are going to be higher generally speaking, as most AVs seem to be imporing some sort of file reputation system. MD and Smartscreen are a great example of this. Between BAFS and Smartscreen, MD/W10 can be pretty ruthless to new files. To the point that it can be extremely annoying for a developer trying to create a new program.

The problem with FPs in tests (at least the way AV-Comparatives displays them) is you only see the total number as part of the graph/chart. AV-Comparatives use to post an appendix to outline in more detail where the FPs came from, as well as if the file had very low, low, medium, or high prevalence. Using MD as an example, it use to have reasonably high FPs in AV-Comparatives. However, when looking at the appendix, one would see that the bulk of the FPs came from files with very low to low prevalence. Files with medium, or high prevalence were extremely low. Which agrees with what the vendors are telling you. Most people in general use software that is fairly well known to most AV vendors, so the FP rate in the real world for the vast majority of people is going to be really low. However, if you are using an AV that is not widely used in your region and you are either using software that is primarily used in that region, or a file with low prevalence in general, your chances of a FP will be higher.


But..but, all those YouTuber tests always showed me that a large folder of random files will just randomly appear on my desktop one day. They also showed me that it's best practice to just execute each one to see what it is, even though it just randomly appeared....:p

You clearly forgot to disable some protection modules as well... specially recommended for ESET and Bitdefender users as their protection modules are interconnected and require each other to offer full protection.
 

Nightwater

Level 2
Jan 26, 2021
69
Nobody can stop this Malware Samurai Crusher while its wearing Iron Man Armor: Mark VII with Captain American' s shield, malware industry is over, period.

02.jpg
Remembering that this shield is made of plastic :ROFLMAO: Malwarebytes is just a second-tier antivirus, I would never trust my system with it
 
F

ForgottenSeer 72227

The easiness of finding big number of malicious files on internet that gets undetected by my AV makes me think why should I even bother using it. That's making the big majority of AVs just a tool to keep average Joe from executing not-a-malware.exe

Your not wrong...there will always be malware available that is undetected. It's the reason why I always say that no AV/IS is ever 100% effective in catching everything, all of the time. It's why I am a person who leans more on education on how to be safe online, instead of putting all of your faith in an AV/IS. Problem is, many people do it the other way around.... put faith in AV and do whatever they want. In saying that....the vast majority of users, regardless of their technical prowess, don't go purposefully looking for malware. So I would argue that in most cases the chances of getting infected are still reasonably low.

Thanks! They had a separate test that I missed.
 

Gandalf_The_Grey

Level 76
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 24, 2016
6,505
 

robboman

Level 2
Verified
Jul 11, 2018
59
Oh no Kaspersky only had a protection rate of 99.96% is this particular test, it's such a failure. I'm gonna be uninstalling Kaspersky right now to switch to another AV that had a protection rate of 99,98%. Lmao

I encountered malware like 1 time during the last 10 years of me browsing the internet so I can't miss those precious 00.02% extra protection 😔

Edit: Panda scoring 99,98% protection rate, seems legit we all know Panda is a beast of a AV
 
Last edited:

blackice

Level 38
Verified
Top Poster
Well-known
Apr 1, 2019
2,731
false positives a bit ghastly??
The FPs are high, but mostly esoteric. I used Windows Defender (when it was called that) through the days of having even worse false positives and never saw a single false positive. I install very few applications regularly outside of games. MB may be moving into the territory of being a decent lightweight solution for low risk users.
 

Nightwalker

Level 24
Verified
Honorary Member
Top Poster
Content Creator
Well-known
May 26, 2014
1,339
The FPs are high, but mostly esoteric. I used Windows Defender (when it was called that) through the days of having even worse false positives and never saw a single false positive. I install very few applications regularly outside of games. MB may be moving into the territory of being a decent lightweight solution for low risk users.

Thats the biggest issue I have with MB, it is not lightweight.
 

blackice

Level 38
Verified
Top Poster
Well-known
Apr 1, 2019
2,731
Thats the biggest issue I have with MB, it is not lightweight.
Really? I just ran a bunch of benchmarks today and it got higher CPU frametimes and framerates than MD, ESET, Bitdefender and definitely better than AVG. I'm wondering if AVG has an issue with nvidia drivers or something because the CPU hit is very surprising. Maybe it's something else on my machine. CPU benchmarks are inline with all the other suites, but any 3D application I take an 8-10% hit on framerate/cpu performance.
 

EndangeredPootis

Level 10
Verified
Well-known
Sep 8, 2019
461
Continuing it here: @EndangeredPootis
Video - Windows Defender vs Malware in 2021 (The PC Security Channel)
It uses Avira's full SDK. So heuristics, cloud, etc all. That's what I meant. So it's normal to see it scoring the same as Avira.
Offline without its cloud? the cloud has the most comprehensive algorithms, just look at MD, it performs absolutely awfully without the cloud, and I highly doubt its all thanks to its signatures.

Also, how does Avira achieve 1% higher online detection rate, despite both supposedly having the same algorithms?
 
  • Like
Reactions: Mercenary and Nevi

SeriousHoax

Level 47
Well-known
Mar 16, 2019
3,630
Offline without its cloud? the cloud has the most comprehensive algorithms, just look at MD, it performs absolutely awfully without the cloud, and I highly doubt its all thanks to its signatures.

Also, how does Avira achieve 1% higher online detection rate, despite both supposedly having the same algorithms?
MD's case is different because they don't push local signatures quick enough. MD's local signature is usually very accurate and they take time to write proper signatures, classify threats appropriately. It's often more than a week behind at creating offline signatures for a new threat. While as I said, Avira is more frequent and release signatures far quicker than Microsoft. Over 10,000 samples were tested and they were a few days old for sure that's why Avira had a high offline detection.
Avira Pro version has a feature like a cloud priority or something. Their pro gets priority over the free version if something not seen before requires cloud analysis. I can't say for sure why there was a difference but TotalAV probably was not quick enough to get a faster cloud response at the time of testing, connectivity issue, or anything. In SE Labs tests, often Avast and AVG had slightly different detection results even though they are the same product. So it could be anything. But the difference is very low so it's nothing to speculate about.
 

Game Of Thrones

Level 5
Verified
Well-known
Jun 5, 2014
220
Really? I just ran a bunch of benchmarks today and it got higher CPU frametimes and framerates than MD, ESET, Bitdefender and definitely better than AVG. I'm wondering if AVG has an issue with nvidia drivers or something because the CPU hit is very surprising. Maybe it's something else on my machine. CPU benchmarks are inline with all the other suites, but any 3D application I take an 8-10% hit on framerate/cpu performance.
mate aside from mb which one of these you recommend? was going to start a thread for this performance thing but saw this. seems you already tested frametime and stuff which one did you see performed better. thanks.
edit: seems your use is like me ordinary stuff and applications web movies and mostly games. appreciate any advice on system performance optimization and program recommendations. will continue on pm to not hijack the thread.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top