AV-Comparatives Real-World Protection Test Feb-Mar 2021 – Factsheet

[Gandalf_The_Grey]

Introduction

Our Real-World Protection Test is currently one of the most comprehensive and complex tests available, using a relatively large number of test cases. Currently, we are running this test under updated Microsoft Windows 10 Pro 64 Bit with up-to-date third-party software (such as Adobe Flash, Adobe Acrobat Reader, Java, etc.). Due to this, finding in-the-field working exploits and running malware is much more challenging than e.g. under a non-up-to-date system with unpatched/vulnerable third-party applications.

This fact sheet is a short overview of the Real-World Protection Test results of February and March 2021. The detailed overall result consumer product reports (covering four months each) are released in June and December. Each of the overall result reports will also contain a false-alarm test and will contain the awards the products reached based on their overall scores during the respective four-month period.

Real-World Protection Test Feb-Mar 2021 - Factsheet

New Consumer Real-World Protection Test Results for Feb-Mar 2021 - Factsheet released!

www.av-comparatives.org

 

[Nightwater]

.

 

[Andy Ful]

All such tests are similar to playing poker or bridge when players have similar playing strengths. The players sometimes can win and sometimes lose. The winner in one particular game is not necessarily the best player and vice versa. Usually, in the next turns the winners and losers change almost randomly. Being the loser in a single game usually does not mean that the playing strength and winning strategy was decreased.

In this particular test, most results except McAfee seem pretty normal as compared to cumulative results. The Avast result is better than usual, but missing 0-3 samples is very common in AV-Comparatives tests (close to the average 4 missed samples for Avast).

AV-Comparatives Real-World 2019-2020 (4 tests) + March 2021
.................... Missed samples....................
----------------------2019-2020----------March 2021
Avast.......................15...................0
Avira........................ 8...................4
Kaspersky Lab.........9....................3
McAfee...................29...................2
Microsoft................10...................7
Norton(Symantec)..5.....................3.5
TrendMicro..............4.....................0

If we sum up the results in the first column then all AVs from the above table missed 80 samples and this gives 20 samples per one test. If we sum up the missed samples in March we get 19.5 samples. So this test on average is similar to all 4 tests from the last two years. These results can also suggest that the infection rate for the average AV in March 2021 is very similar to the years 2019-2021.

The results of Microsoft and Norton are worse than usual but still, this can be an accidental result (especially for Norton). In the case of Microsoft, it can be also a suggestion that in the testing period the Machine Learning algorithm was not optimal. This is a constant problem for many AVs to keep the right balance between protection and false positives rate.

Post edited (added comment for Avast).