AV-Comparatives Malware Protection Test September 2021

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Gandalf_The_Grey

Level 76
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 24, 2016
6,506
Introduction
In the Malware Protection Test, malicious files are executed on the system. While in the Real-World Protection Test the vector is the web, in the Malware Protection Test the vectors can be e.g. network drives, USB or cover scenarios where the malware is already on the disk.

Please note that we do not recommend purchasing a product purely on the basis of one individual test or even one type of test. Rather, we would suggest that readers consult also our other recent test reports, and consider factors such as price, ease of use, compatibility and support. Installing a free trial version allows a program to be tested in everyday use before purchase.

In principle, home-user Internet security suites are included in this test. However, some vendors asked us to include their (free) antivirus security product instead.
Offline vs. Online Detection Rates
Many of the products in the test make use of cloud technologies, such as reputation services or cloud-based signatures, which are only reachable if there is an active Internet connection. By performing on-demand and on-access scans both offline and online, the test gives an indication of how cloud-dependent each product is, and consequently how well it protects the system when an Internet connection is not available. We would suggest that vendors of highly cloud-dependent products should warn users appropriately in the event that the connectivity to the cloud is lost, as this may considerably affect the protection provided. While in our test we check whether the cloud services of the respective security vendors are reachable, users should be aware that merely being online does not necessarily mean that their product’s cloud service is reachable/working properly.

For readers’ information and due to frequent requests from magazines and analysts, we also indicate how many of the samples were detected by each security program in the offline and online detection scans.
Schermafbeelding 2021-10-15 200108.jpg
Test Results
Schermafbeelding 2021-10-15 200333.jpg
False Positive (False Alarm) Test Result
Schermafbeelding 2021-10-15 200533.jpg

Details about the discovered false alarms (including their assumed prevalence) can be seen in the separate report available at: https://www.av-comparatives.org/wp-content/uploads/2021/10/avc_fps_202109.pdf
 

ExecutiveOrder

Level 2
Sep 21, 2021
47
Pleasantly surprised at Microsoft's result.

What happened to ESET...
Yeah, looking at 2-year data:
- March'20, MS scored 99.88% (3rd cluster) of protection, 9 FPs ('few'), Standard Award.
- Sept'20, MS scored 100% at cost of 21 FPs ('many'), Advanced Award.
- March'21, looks like MS tried its best to gain low FPs (4, 'few') but ends up in 99.85% (3rd cluster) of protection, Standard Award.
- And now it successfully gets a nice 99.97% protection (1st cluster) with only a false positive (best, tied with ESET), Advanced+.
Congrats to Microsoft.

For ESET, not sure what happened, I think they struggle to keep the balance this year despite false positives results getting even better and the best in the chart.
FP gets even lower (0) in March'21 test compared to last year (2) but the protection degraded a bit (2nd cluster, Advanced Award, previously +), now it still gets lower FP (1) compared to the previous year (2) but the protection rate also degraded even further a bit (placed in 3rd cluster, Standard Award).
ESET last Real-World Test not looking well too, only Standard Award, unlike the previous test.
I think the second (and last of this year) test of the Real-World Test is going to be released in a month from now, hope they will get back into the top chart.

Kaspersky just like the last "Real-World Test" result, protection degraded a bit and didn't manage to get Advanced+ anymore, it is also good when it comes to FP.
Looks like Bitdefender going to overthrow Kaspersky and reclaim the Product of the Year award this year. Bitdefender is the only vendor with Advanced+ in the last Real World Test this year and keeps getting Advanced+ in Malware-Protection and Performance test so far.
 

Anthony Qian

Level 9
Verified
Well-known
Apr 17, 2021
448
Yeah, looking at 2-year data:
- March'20, MS scored 99.88% (3rd cluster) of protection, 9 FPs ('few'), Standard Award.
- Sept'20, MS scored 100% at cost of 21 FPs ('many'), Advanced Award.
- March'21, looks like MS tried its best to gain low FPs (4, 'few') but ends up in 99.85% (3rd cluster) of protection, Standard Award.
- And now it successfully gets a nice 99.97% protection (1st cluster) with only a false positive (best, tied with ESET), Advanced+.
Congrats to Microsoft.

For ESET, not sure what happened, I think they struggle to keep the balance this year despite false positives results getting even better and the best in the chart.
FP gets even lower (0) in March'21 test compared to last year (2) but the protection degraded a bit (2nd cluster, Advanced Award, previously +), now it still gets lower FP (1) compared to the previous year (2) but the protection rate also degraded even further a bit (placed in 3rd cluster, Standard Award).
ESET last Real-World Test not looking well too, only Standard Award, unlike the previous test.
I think the second (and last of this year) test of the Real-World Test is going to be released in a month from now, hope they will get back into the top chart.

Kaspersky just like the last "Real-World Test" result, protection degraded a bit and didn't manage to get Advanced+ anymore, it is also good when it comes to FP.
Looks like Bitdefender going to overthrow Kaspersky and reclaim the Product of the Year award this year. Bitdefender is the only vendor with Advanced+ in the last Real World Test this year and keeps getting Advanced+ in Malware-Protection and Performance test so far.
BTW, ESET got the lowest protection rate in the latest AV-C Business antivirus test (https://malwaretips.com/threads/business-security-test-august-september-2021-–-factsheet.110486/)
 

dabluez98

Level 3
Verified
Oct 2, 2018
138
@Anthony, I wonder with the newer protection Kaspersky 2022 will have, if this will make any difference. I am quoting this from Kaspersky employee, not sure if its just hype/new marketing features or new real features. But it seems their consumer stuff coming later than usual this year as it says.

"
Hi All!!

Kaspersky’s consumer products update will be available later than usual as we are preparing for something special and large-scale. We are currently beta-testing some of our future product variants among users so that we can provide a brand-new product experience with a redesigned interface and new features, which will be released next year. – attributable to Kaspersky.

 
  • Like
Reactions: amico81 and Nevi

Anthony Qian

Level 9
Verified
Well-known
Apr 17, 2021
448
@Anthony, I wonder with the newer protection Kaspersky 2022 will have, if this will make any difference. I am quoting this from Kaspersky employee, not sure if its just hype/new marketing features or new real features. But it seems their consumer stuff coming later than usual this year as it says.

"
Hi All!!



The next version of Kaspersky consumer products is said to have a brand new interface! :sneaky: But tech wise, I don't think they will add some features like cloud sandboxing to it.
 
  • Like
Reactions: Nevi and dabluez98

tipo

Level 8
Well-known
Jul 26, 2012
353
Great job Norton! 🥳
Oh, yeah, I forgot to mention I'm using Norton LifeLock. 😬 I don't mind the false positives, better with higher fps than lower detection rate.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top