AV-Comparatives Business Security Test August-September 2021 – Factsheet

[Gandalf_The_Grey]

Introduction
This is a short fact sheet for our Business Main-Test Series, containing the results of the Business Malware Protection Test (September) and Business Real-World Protection Test (August-September). The full report, including the Performance Test and product reviews, will be released in December. To be certified in December 2021 as an “Approved Business Product” by AV-Comparatives, the tested products must score at least 90% in the Malware Protection Test with zero false alarms on common business software, and at least 90% in the overall Real-World Protection Test (i.e. over the course of four months), with less than one hundred false alarms on any clean software/websites (and with zero false alarms on common business software). Tested products must also avoid major performance issues (impact score must be below 40) and have fixed all reported bugs in order to gain certification.

Please note that the results of the Business Main-Test Series cannot be compared with the results of the Consumer Main-Test Series, as the tests are done at different times, with different sets, different settings, etc.

Starting from 2022, products will be required to have an FP rate on non-business files below the Remarkably High threshold.

Below we have listed relevant deviations from default settings (i.e. setting changes applied by the vendors):

Acronis: “Backup”, “Vulnerability assessment”, “Patch management” and “Data protection map” disabled.

Bitdefender: “Fileless Attack Protection”, “Sandbox Analyzer” (for Applications and Documents) and “Scan SSL” enabled. “Encryption” and “Patch Management” add-ons registered and enabled. “HyperDetect” and “Device Sensor” disabled. “Update ring” changed to “Fast ring”. “Web Traffic Scan” enabled for HTTP Web traffic and Incoming POP3 emails.

Cisco: “On Execute File and Process Scan” set to Active; “Exploit Prevention: Script Control” and “TETRA Deep Scan File” enabled; “Event Tracing for Windows” enabled.

CrowdStrike: everything enabled and set to maximum, i.e. “Extra Aggressive”. “Sensor Visibility” for “Firmware” disabled. Uploading of “Unknown Detection-Related Executables” and “Unknown Executables” disabled.

Cybereason: “Anti-Malware” enabled; “Signatures mode” set to “Disinfect”; “Behavioral document protection” enabled; “Artificial intelligence” and “Anti-Exploit” set to “Aggressive”; “Exploit protection”, “PowerShell and .NET”, “Anti-Ransomware” and “App Control” enabled and set to “Prevent”; all “Collection features” enabled; “Scan archives on access” enabled.

Elastic: MalwareScore (“windows.advanced.malware.threshold”) set to “aggressive”.

ESET: All “Real-Time & Machine Learning Protection” settings set to “Aggressive”.

FireEye: “Real-Time Indicator Detection” disabled, “Exploit Guard” and “Malware Protection” enabled.

Fortinet: “Sandbox analysis” (FortiSandbox) and FortiEDR enabled. “Submit files from USB Sources” disabled; “Exclude Files from Trusted Sources” for “Sandbox Detection” enabled; in “Execution Prevention”, “Suspicious Script Execution” was disabled and “Unconfirmed File Detected” was enabled; eXtended Detection (XDR) was disabled.

G Data: “BEAST Behavior Monitoring” set to “Halt program and move to quarantine”. “G DATA WebProtection” add-on for Google Chrome installed and activated.

Malwarebytes: “Expert System Algorithms”, “Block penetration testing attacks”, “Disable IE VB Scripting”, “Java Malicious Inbound/outbound Shell Protection”, “Earlier RTP blocking”, “Enhanced sandbox protection” and “Thorough scan” enabled; “RET ROP Gadget detection” and “Malicious LoadLibrary Protection” enabled for all applications; “Protection for MessageBox Payload” enabled for MS Office; “Malwarebytes Browser Guard” Chrome extension enabled.

Microsoft: Google Chrome extension “Windows Defender Browser Protection” installed and enabled.

Sophos: “Threat Case creation” and “Web Control” disabled.

VIPRE: “DNS Traffic Filtering” and “Malicious URL Blocking for HTTPS Traffic” enabled. “Firewall” and “IDS” enabled and set to “Block With Notify”.

VMware: policy set to “Advanced”.

Avast, K7, Kaspersky, Panda: default settings.

Test Results

Real-World Protection Test (August-September)

Malware Protection Test (September)

Business Security Test August-September 2021 - Factsheet

The Business Security Test August-September 2021 - Factsheet covering results of our Enterprise main-test series has been released.

www.av-comparatives.org

 

[Anthony Qian]

OMG, ESET ranked last for protection rate.

 

[Nightwalker]

OMG, ESET ranked last for protection rate.

For perspective, this means 18 malwares missed in 1016 samples for Business Malware Protection Test, Kaspersky for example with 99 % missed 10 samples; I dont think that there is any reason for concern.

 

[Anthony Qian]

For perspective, this means 18 malwares missed in 1016 samples for Business Malware Protection Test, Kaspersky for example with 99 % missed 10 samples; I dont think that there is any reason for concern.

Microsoft missed only 1 sample; Avast and Bitdefender missed only 2 samples! I think relative performance is more important in a test.

 

[Nightwalker]

Microsoft missed only 1 sample; Avast and Bitdefender missed only 2 samples! I think relative performance is more important in a test.

It just means that in this test with those specific samples those antivirus solutions detected more, statistically it doesnt mean that ESET is a bad product incapable of doing its job, because 98,2 % is still quite good.

There is no reason to concern at all, except that if you care that the antivirus you are using is ranked first in lab tests.

 

[printing]

How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.

 

[Nightwalker]

How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.

I dont think that it is enough, you can use Gartner Magic Quadrant for a more insightful and complementary report.

https://www.gartner.com/doc/reprints?id=1-2435Z2CX&ct=200903&st=sb

Gartner names Microsoft a Leader in the 2021 Endpoint Protection Platforms Magic Quadrant | Microsoft Security Blog

Gartner names Microsoft a Leader in 2021 Endpoint Protection Platforms (EPP) Magic Quadrant, positioned highest on the ability to execute.

www.microsoft.com

 

[Anthony Qian]

It just means that in this test with those specific samples those antivirus solutions detected more, statistically it doesnt mean that ESET is a bad product incapable of doing its job, because 98,2 % is still quite good.

There is no reason to concern at all, except that if you care that the antivirus you are using is ranked first in lab tests.

Based on other products' performance, I don't think this test is hard. ESET, given its high price, is supposed to do better.

 

[Anthony Qian]

How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.

If you don't trust AV-C, then look at AV-Test result. Microsoft Defender is just a nice solution that keeps getting better. With such a large user base, I am confident that its performance will continue to improve.

 

[roger_m]

Based on other products' performance, I don't think this test is hard. ESET, given its high price, is supposed to do better.

You should look at the results of multiple tests, rather than just one test.

 

[Anthony Qian]

You should look at the results of multiple tests, rather than just one test.

Actually I did.

1. Since the beginning of this year, ESET's home product has been declining in the AV-C Malware Protection Test. (ESET )
2. Since the beginning of the year, ESET has performed poorly in AV-Test tests in terms of Protection.(Test antivirus software ESET)
3. According to SE Labs Enterprise Endpoint Protection (2021 Q2), ESET came in last place in terms of protection. (Enterprise Endpoint Protection (2021 Q2) - SE Labs - Reports)

 

[The_King]

How can i convince my boss that the Microsoft Defender is good enough over a paid vendor?
Is quoting the report good enough?
I doubt any companies would take the risk and be blame for not choosing a paid vendor if a pc is compromised.

If you are running a small business without a network then you may get away with just an AV solution.

If the business is running on a network architecture then you should have an Endpoint AV in place
to manage security on users PC. All it would take is one user to open a compromised email attachment and the whole network
could be compromised with Ransomware or malware.

 

[printing]

If you don't trust AV-C, then look at AV-Test result. Microsoft Defender is just a nice solution that keeps getting better. With such a large user base, I am confident that its performance will continue to improve.

Okay. thanks for the link.

 

[printing]

I dont think that it is enough, you can use Gartner Magic Quadrant for a more insightful and complementary report.

https://www.gartner.com/doc/reprints?id=1-2435Z2CX&ct=200903&st=sb

Gartner names Microsoft a Leader in the 2021 Endpoint Protection Platforms Magic Quadrant | Microsoft Security Blog

Gartner names Microsoft a Leader in 2021 Endpoint Protection Platforms (EPP) Magic Quadrant, positioned highest on the ability to execute.

www.microsoft.com

wow!! thanks !!!