AV-Comparatives Business Security Test 2021 (March – June)

[Gandalf_The_Grey]

Introduction
his is the first half-year report of our Business Main-Test Series of 2021, containing the results of the Business Real-World Protection Test (March-June), Business Malware Protection Test (March), Business Performance Test (June), as well as the Product Reviews.

Please note that the results of the Business Main-Test Series cannot be compared with the results of the Consumer Main-Test Series, as the tests are done at different times, with different sets, different settings, etc.

Test Procedure
The test series consists of three main parts:

The Real-World Protection Test mimics online malware attacks that a typical business user might encounter when surfing the Internet.

The Malware Protection Test considers a scenario in which the malware pre-exists on the disk or enters the test system via e.g. the local area network or removable device, rather than directly from the Internet.

In addition to each of the protection tests, a False-Positives Test is conducted, to check whether any products falsely identify legitimate software as harmful.

The Performance Test looks at the impact each product has on the system’s performance, i.e. how much it slows down normal use of the PC while performing certain tasks.

To complete the picture of each product’s capabilities, there is a user-interface review included in the report as well.

Real-World Protection Test (March-June)

The results below are based on a test set consisting of 759 test cases (such as malicious URLs), tested from the beginning of March 2021 till the end of June 2021.
CrowdStrike, Panda, Fortinet, Malwarebytes and Cybereason had above-average numbers of FPs (on non-business software) in the Real-World Protection Test. Cisco had one false positive on common business software.

Malware Protection Test (March)

Performance Test (May – June)
Summarized results
Users should weight the various subtests according to their needs. We applied a scoring system to sum up the various results. Please note that for the File Copying and Launching Applications subtests, we noted separately the results for the first run and for subsequent runs. For the AV-C score, we took the rounded mean values of first and subsequent runs for File Copying, whilst for Launching Applications we considered only the subsequent runs. “Very fast” gets 15 points, “fast” gets 10 points, “mediocre” gets 5 points and “slow” gets 0 points. This leads to the following results:

Business Security Test 2021 (March - June)

The first half-year report of our Enterprise main-test series containing a Real-World Protection, Malware Protection and Performance Test has been released. Also product reviews are included. Read the report for details.

www.av-comparatives.org

 

[SeriousHoax]

The chart/table that is showing here in the post is not accurate. The table is a bit out of the place so one should visit the link to check the full and accurate result.

 

[Gandalf_The_Grey]

The chart/table that is showing here in the post is not accurate. The table is a bit out of the place so one should visit the link to check the full and accurate result.

Which chart is that?

 

[SeriousHoax]

Which chart is that?

Check the first chart as an example. They are out of place. The UserDependent section is actually the Blocked section. Like this, every chart is not in its correct place.

Blocked

Userdependent

Compromised

 

[Gandalf_The_Grey]

Check the first chart as an example. They are out of place. The UserDependent section is actually the Blocked section. Like this, every chart is not in its correct place.

Blocked

Userdependent

Compromised

Ah, I see, thanks, difficulties with copy pasting...
I will make some screenshots, that will work better

 

[SeriousHoax]

Ah, I see, thanks, difficulties with copy pasting...
I will make some screenshots, that will work better

Yeah, I have had similar difficulties in copying too. It often doesn't show up the way it should.

 

[Andy Ful]

The last 6 AV-Comparatives Business tests (including the March-June 2021) for popular AVs:

Real-World (1207+732+844+767+801+759 samples)

------------------------ Missed samples-----------------
Bitdefender.......0+0+0+4+2+1.........=7
Panda................5+4+0+0+1+1.........=11
Kaspersky.........3+0+4+1+1+3.........=12
VIPRE................3+1+4+4+2+1.........=15
Microsoft.......(10)+(2)+(7)+2+2+2..=15.5
Avast.............4+4+12+1+1+(1)3........=25.5
ESET.................13+8+4+3+2+6.........=36
Sophos........?+2+4+4+14+(2)16........=*49 (first scoring averaged)


Malware Protection (1556+1311+1278+1192+1603+1008 samples)

------------------------ Missed samples -----------------
Avast................0+1+0+2+0+0........=3
Bitdefender......0+1+3+1+2+0........=7
Microsoft.........2+7+1+0+0+4........=14
Panda.............2+1+1+1+13+1.......=19
VIPRE.............0+10+4+2+6+1.......=23
Sophos............?+1+3+7+5+4........=*24 (first scoring averaged)
Kaspersky.......3+13+8+6+5+4.......=39
ESET...............2+20+9+1+2+7.......=41

Avast Business Antivirus Pro Plus 20.10 21.2
Bitdefender GravityZone Elite 6.6 6.6
ESET PROTECT Entry with ESET PROTECT Cloud 8.0 8.0
Kaspersky Endpoint Security for Business – Select, with KSC 11.5 11.5
Microsoft Defender & Microsoft Endpoint Manager 4.18 4.18 + Windows Defender Browser Protection
Panda Endpoint Protection Plus on Aether 8.0 8.0
Sophos Intercept X Advanced 10.8 10.8
Vipre Endpoint Cloud 12.0 12.0

Many AVs have used tweaked settings - here are the settings used in the March-June 2021:

Bitdefender: “Fileless Attack Protection”, “Sandbox Analyzer” (for Applications and Documents) and “Scan SSL” enabled. “Encryption” and “Patch Management” add-ons registered and enabled. “HyperDetect” and “Device Sensor” disabled. “Update ring” changed to “Fast ring”. “Web Traffic Scan” enabled for HTTP Web traffic and Incoming POP3 emails.

ESET: All “Real-Time & Machine Learning Protection” settings set to “Aggressive”.

Microsoft: Google Chrome extension “Windows Defender Browser Protection” installed and enabled; “CloudBlockLevel” set to “High”.

Sophos: “Threat Case creation” and “Web Control” disabled.

Vipre: “DNS Traffic Filtering” and “Malicious URL Blocking for HTTPS Traffic” enabled. “Firewall” and “IDS” enabled and set to “Block With Notify”.

Avast, Kaspersky, Panda: default settings.


Edit.
The Malware Protection tests are less important because most of the attacks in the wild that include these samples are already prevented by AVs before the sample could be executed (network protection + web protection + email protection, etc).