AV-Comparatives Business Security Test August-September 2021 – Factsheet

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Gandalf_The_Grey

Level 53
Verified
Trusted
Content Creator
Apr 24, 2016
4,200
40,832
Introduction
This is a short fact sheet for our Business Main-Test Series, containing the results of the Business Malware Protection Test (September) and Business Real-World Protection Test (August-September). The full report, including the Performance Test and product reviews, will be released in December. To be certified in December 2021 as an “Approved Business Product” by AV-Comparatives, the tested products must score at least 90% in the Malware Protection Test with zero false alarms on common business software, and at least 90% in the overall Real-World Protection Test (i.e. over the course of four months), with less than one hundred false alarms on any clean software/websites (and with zero false alarms on common business software). Tested products must also avoid major performance issues (impact score must be below 40) and have fixed all reported bugs in order to gain certification.

Please note that the results of the Business Main-Test Series cannot be compared with the results of the Consumer Main-Test Series, as the tests are done at different times, with different sets, different settings, etc.

Starting from 2022, products will be required to have an FP rate on non-business files below the Remarkably High threshold.
Below we have listed relevant deviations from default settings (i.e. setting changes applied by the vendors):

Acronis:
“Backup”, “Vulnerability assessment”, “Patch management” and “Data protection map” disabled.

Bitdefender: “Fileless Attack Protection”, “Sandbox Analyzer” (for Applications and Documents) and “Scan SSL” enabled. “Encryption” and “Patch Management” add-ons registered and enabled. “HyperDetect” and “Device Sensor” disabled. “Update ring” changed to “Fast ring”. “Web Traffic Scan” enabled for HTTP Web traffic and Incoming POP3 emails.

Cisco: “On Execute File and Process Scan” set to Active; “Exploit Prevention: Script Control” and “TETRA Deep Scan File” enabled; “Event Tracing for Windows” enabled.

CrowdStrike: everything enabled and set to maximum, i.e. “Extra Aggressive”. “Sensor Visibility” for “Firmware” disabled. Uploading of “Unknown Detection-Related Executables” and “Unknown Executables” disabled.

Cybereason: “Anti-Malware” enabled; “Signatures mode” set to “Disinfect”; “Behavioral document protection” enabled; “Artificial intelligence” and “Anti-Exploit” set to “Aggressive”; “Exploit protection”, “PowerShell and .NET”, “Anti-Ransomware” and “App Control” enabled and set to “Prevent”; all “Collection features” enabled; “Scan archives on access” enabled.

Elastic: MalwareScore (“windows.advanced.malware.threshold”) set to “aggressive”.

ESET: All “Real-Time & Machine Learning Protection” settings set to “Aggressive”.

FireEye: “Real-Time Indicator Detection” disabled, “Exploit Guard” and “Malware Protection” enabled.

Fortinet: “Sandbox analysis” (FortiSandbox) and FortiEDR enabled. “Submit files from USB Sources” disabled; “Exclude Files from Trusted Sources” for “Sandbox Detection” enabled; in “Execution Prevention”, “Suspicious Script Execution” was disabled and “Unconfirmed File Detected” was enabled; eXtended Detection (XDR) was disabled.

G Data: “BEAST Behavior Monitoring” set to “Halt program and move to quarantine”. “G DATA WebProtection” add-on for Google Chrome installed and activated.

Malwarebytes: “Expert System Algorithms”, “Block penetration testing attacks”, “Disable IE VB Scripting”, “Java Malicious Inbound/outbound Shell Protection”, “Earlier RTP blocking”, “Enhanced sandbox protection” and “Thorough scan” enabled; “RET ROP Gadget detection” and “Malicious LoadLibrary Protection” enabled for all applications; “Protection for MessageBox Payload” enabled for MS Office; “Malwarebytes Browser Guard” Chrome extension enabled.

Microsoft: Google Chrome extension “Windows Defender Browser Protection” installed and enabled.

Sophos: “Threat Case creation” and “Web Control” disabled.

VIPRE: “DNS Traffic Filtering” and “Malicious URL Blocking for HTTPS Traffic” enabled. “Firewall” and “IDS” enabled and set to “Block With Notify”.

VMware: policy set to “Advanced”.

Avast, K7, Kaspersky, Panda: default settings.
Test Results
Real-World Protection Test (August-September)
Schermafbeelding 2021-10-15 201551.jpg
Malware Protection Test (September)
Schermafbeelding 2021-10-15 201709.jpg

Schermafbeelding 2021-10-15 201849.jpgSchermafbeelding 2021-10-15 201910.jpg
 

Anthony Qian

Level 3
Apr 17, 2021
148
745
For perspective, this means 18 malwares missed in 1016 samples for Business Malware Protection Test, Kaspersky for example with 99 % missed 10 samples; I dont think that there is any reason for concern.
Microsoft missed only 1 sample; Avast and Bitdefender missed only 2 samples! I think relative performance is more important in a test.
 

Nightwalker

Level 22
Verified
Trusted
Content Creator
May 26, 2014
1,187
7,881
Microsoft missed only 1 sample; Avast and Bitdefender missed only 2 samples! I think relative performance is more important in a test.

It just means that in this test with those specific samples those antivirus solutions detected more, statistically it doesnt mean that ESET is a bad product incapable of doing its job, because 98,2 % is still quite good.

There is no reason to concern at all, except that if you care that the antivirus you are using is ranked first in lab tests.
 

printing

Level 1
Nov 14, 2020
23
32
How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.
 

Nightwalker

Level 22
Verified
Trusted
Content Creator
May 26, 2014
1,187
7,881
How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.
I dont think that it is enough, you can use Gartner Magic Quadrant for a more insightful and complementary report.

 

Anthony Qian

Level 3
Apr 17, 2021
148
745
It just means that in this test with those specific samples those antivirus solutions detected more, statistically it doesnt mean that ESET is a bad product incapable of doing its job, because 98,2 % is still quite good.

There is no reason to concern at all, except that if you care that the antivirus you are using is ranked first in lab tests.
Based on other products' performance, I don't think this test is hard. ESET, given its high price, is supposed to do better.
 

Anthony Qian

Level 3
Apr 17, 2021
148
745
How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.
If you don't trust AV-C, then look at AV-Test result. Microsoft Defender is just a nice solution that keeps getting better. With such a large user base, I am confident that its performance will continue to improve.
 

Anthony Qian

Level 3
Apr 17, 2021
148
745
You should look at the results of multiple tests, rather than just one test.
Actually I did.
  1. Since the beginning of this year, ESET's home product has been declining in the AV-C Malware Protection Test. (ESET )
  2. Since the beginning of the year, ESET has performed poorly in AV-Test tests in terms of Protection.(Test antivirus software ESET)
  3. According to SE Labs Enterprise Endpoint Protection (2021 Q2), ESET came in last place in terms of protection. (Enterprise Endpoint Protection (2021 Q2) - SE Labs - Reports)
 

The_King

Level 11
Verified
Aug 2, 2020
543
5,962
How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.
If you are running a small business without a network then you may get away with just an AV solution.

If the business is running on a network architecture then you should have an Endpoint AV in place
to manage security on users PC. All it would take is one user to open a compromised email attachment and the whole network
could be compromised with Ransomware or malware.
 

printing

Level 1
Nov 14, 2020
23
32
I dont think that it is enough, you can use Gartner Magic Quadrant for a more insightful and complementary report.

wow!! thanks !!!
 
Top