AV-Comparatives Consumer Real-World Protection Test July-August 2021 – Factsheet

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Gandalf_The_Grey

Level 76
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 24, 2016
6,506
Introduction

Our Real-World Protection Test is currently one of the most comprehensive and complex tests available, using a relatively large number of test cases. Currently, we are running this test under updated Microsoft Windows 10 Pro 64 Bit with up-to-date third-party software. Due to this, finding in-the-field working exploits and running malware is much more challenging than e.g. under a non-up-to-date system with unpatched/vulnerable third-party applications.

This Real-World Protection Test factsheet is a short overview of the Real-World Protection Test results of July and August 2021. The detailed overall result consumer product reports (covering four months each) are released in June and November. Each of the overall result reports will also contain a false-alarm test and will contain the awards the products reached based on their overall scores during the respective four-month period.
Test Results

The results are based on the test set of 380 live test cases (malicious URLs found in the field), consisting of working exploits (i.e. drive-by downloads) and URLs pointing directly to malware. Thus, exactly the same infection vectors are used as a typical user would experience in everyday life. The test-cases used cover a wide range of current malicious sites and provide insights into the protection given by the various products (using all their protection features) while surfing the web.

The following products (latest version available at time of testing) were tested: Avast Free Antivirus, AVG Free Antivirus, Avira Antivirus Pro, Bitdefender Internet Security, ESET Internet Security, G Data Total Security, K7 Total Security, Kaspersky Internet Security, Malwarebytes Premium, McAfee Total Security, Microsoft Defender, Panda Free Antivirus, NortonLifeLock Norton 360, TotalAV Total Security, Total Defense Essential Anti-Virus, Trend Micro Internet Security and Vipre Advanced Security.

Every few months we update the charts on our website showing the protection rates of the various tested products over the various months. The interactive charts can be found on our website. The chart below shows only the protection scores for the months of JULY and AUGUST 2021 (380 test cases). The results of the false-positives test are also shown in the graph below.
Schermafbeelding 2021-09-15 124211.png
 

The_King

Level 12
Verified
Top Poster
Well-known
Aug 2, 2020
542
Another excellent result for MD/WD with 0 FP. Too bad about the recent Zloader attacks, that disables it and infects users.

Still, I'm sure things will improve, and soon I may say goodbye to BD in the not too distant future.
 

plat

Level 29
Top Poster
Sep 13, 2018
1,793
I get this little "jolt" or something when I hear about McAfee--I guess it's because the founder was so uh, colorful and the product underwent so many changes. I just said somewhere the antivirus has quietl;y become respectable and this lab test supports it--at least for this quarter.

Is it still so difficult to remove nowadays?
 
F

ForgottenSeer 92963

@Reiner and @amico81

I worked for AV-C (retired now). Although I am not a security expert, I can assure you the samples of AV-C are usually zero day/zero hour. For what I know, reputation scoring and context related data point analysis make a difference when malware is executed from disk or downloaded/executed via a webbrowser (which the real world tests simulate). The executing from disk might explain these differences.

1631727115386.png
 

SeriousHoax

Level 47
Well-known
Mar 16, 2019
3,630
For what I know, reputation scoring and context related data point analysis make a difference when malware is executed from disk or downloaded/executed via a webbrowser (which the real world tests simulate). The executing from disk might explain these differences.
This is what I've said a couple of times in the forum. Also mentioned by malware analyst @struppigel in the past. But some forum members tend often forget/ignore this.
This downloaded/executed via a web browser factor is likely the reason why Trend and Panda most of the time do well in the Real-World tests. But both of these products show comparatively worst results in the Malware Protection tests where malware are executed from the disk.
 

Gandalf_The_Grey

Level 76
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 24, 2016
6,506
So, you should compare the HUB results with AV-Comparatives Malware Protection Test.
In the Malware Protection Test, malicious files are executed on the system. While in the Real-World Protection Test the vector is the web, in the Malware Protection Test the vectors can be e.g. network drives, USB or cover scenarios where the malware is already on the disk.
Posted and discussed here:
 

Anthony Qian

Level 9
Verified
Well-known
Apr 17, 2021
448
At AV-Comparatives Trend Micro is unbeatable, always the best protection, here in the Hub it's almost always infected, bizarre :unsure:
If you look at AV-C's Malware Protection Test March 2021, you will find Trend Micro did a poor job. Indeed, TM is good at blocking online threats but not good at scanning and behavioral blocking.

So, you should compare the HUB results with AV-Comparatives Malware Protection Test.

Posted and discussed here:
True. AV-C's Real-World Protection Test is mainly focused on online threats.
 
Last edited:

cruelsister

Level 42
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 13, 2013
3,133
Too bad about the recent Zloader attacks, that disables it and infects users.
The malware actually will not Disable WD totally (real Time stays enabled), but is prettier in that only the Cloud functionality as well as sample submission are disabled. This allows for a true zero day file to get through as well as staying undetectable for a bit as Microsoft will remain oblivious to it
 

SeriousHoax

Level 47
Well-known
Mar 16, 2019
3,630
True for most av-products here, but the tested panda free doesnt have any web protection. That's curious
What he meant is, some AVs are more sensitive to files downloaded from the internet and also downloaded files that are executed directly from the browser unlike a file that's already on the disk or something like you extract files from a zip file using 7zip.
 

Gandalf_The_Grey

Level 76
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 24, 2016
6,506
What he meant is, some AVs are more sensitive to files downloaded from the internet and also downloaded files that are executed directly from the browser unlike a file that's already on the disk or something like you extract files from a zip file using 7zip.
The famous mark of the web:
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top