AV-Comparatives Business Security Test 2020 (August – November)

[Gandalf_The_Grey]

Introduction

This is the second half-year report of our Business Main-Test Series of 2020, containing the results of the Business Real-World Protection Test (August-November), Business Malware Protection Test (September), Business Performance Test (November), as well as the Product Reviews.

Information about additional third-party engines/signatures used by some of the products: Acronis, Cisco, Cybereason, FireEye, G Data and Vipre use the Bitdefender engine (in addition to their own protection features). VMware uses the Avira engine (in addition to their own protection features). G Data’s OutbreakShield is based on Cyren.

ESET Endpoint Protection Advanced Cloud has recently been renamed in ESET PROTECT Entry.

We congratulate the vendors who are participating in the Business Main-Test Series for having their business products publicly tested by an independent lab, showing their commitment to improving their products, being transparent to their customers and having confidence in their product quality.

Test Procedure

The test series consists of three main parts:

The Real-World Protection Test mimics online malware attacks that a typical business user might encounter when surfing the Internet.

The Malware Protection Test considers a scenario in which the malware pre-exists on the disk or enters the test system via e.g. the local area network or removable device, rather than directly from the Internet.

In addition to each of the protection tests, a False-Positives Test is conducted, to check whether any products falsely identify legitimate software as harmful.

The Performance Test looks at the impact each product has on the system’s performance, i.e. how much it slows down normal use of the PC while performing certain tasks.

To complete the picture of each product’s capabilities, there is a user-interface review included in the report as well.

Some of the products in the test are clearly aimed at larger enterprises and organisations, while others are more applicable to smaller businesses. Please see each product’s review section for further details.

Business Security Test 2020 (August - November)

The second half-year report of our new Enterprise main-test series containing a Real-World Protection, Malware Protection and Performance Test has been released. Also product reviews are included. Read the report for details.

www.av-comparatives.org

 

[FireHammer]

Hi @Gandalf_The_Grey, as far as I can see(read)Bitdefender & Kaspersky is the winners of this test with Microsoft as runner up, Avast did well also.
what do you think? Eset & K7 did amazingly well!

 

[Gandalf_The_Grey]

Hi @Gandalf_The_Grey, as far as I can see(read)Bitdefender & Kaspersky is the winners of this test with Microsoft as runner up, Avast did well also.
what do you think? Eset & K7 did amazingly well!

It's complicated because there is no overall score if I read the article correctly.

Real-World Protection:

Blocked	User dependent	Compromised	PROTECTION RATE [Blocked % + (User dependent %)/2]*	False Alarms
Kaspersky	800	–	1	99.9%	2
Avast	800	–	1	99.9%	8
Fortinet	800	–	1	99.9%	10
Panda	800	–	1	99.9%	32
ESET	799	–	2	99.8%	1
VIPRE	799	–	2	99.8%	3
Bitdefender, Microsoft	799	–	2	99.8%	5
K7	797	–	4	99.5%	29
G Data	796	–	5	98.4%	8
Elastic	791	–	10	98.8%	17
Sophos	787	–	14	98.3%	4
SparkCognition	785	–	16	98.0%	3
Crowdstrike	783	–	18	97.8%	20
Acronis	782	–	19	97.6%	1
VMWare	782	–	19	97.6%	4
Cisco	772	–	29	96.4%	2
FireEye	765	–	36	95.5%	2
Cybereason*	734	–	67	91.6%	32

Malware Protection and false alarms:

Malware Protection Rate	False Alarms on common business software
Avast, Fortinet, G Data, Microsoft, VMware	100%	0
Bitdefender, ESET, K7	99.9%	0
Cisco, FireEye, Kaspersky, Sophos	99.7%	0
Crowdstrike, Vipre	99.6%	0
Cybereason, Elastic	99.5%	0
Acronis	99.4%	0
SparkCognition	99.3%	0
Panda	99.2%	0

Performance:

AVC Score	PC Mark Score	Impact Score
1.	ESET, K7	90	98.8	1.2
2.	Elastic	90	98.5	1.5
3.	Kaspersky	90	97.6	2.4
4.	Bitdefender	90	97.4	2.6
5.	Avast	90	96.7	3.3
6.	CrowdStrike, Panda	90	96.4	3.6
7.	VMware	85	96.3	8.7
8.	SparkCognition	80	97.3	12.7
9.	VIPRE	80	97.1	12.9
10.	FireEye	80	96.8	13.2
11.	G DATA	78	96.3	15.7
12.	Acronis, Cisco	75	97.9	17.1
13.	Sophos	75	97.6	17.4
14.	Microsoft	75	96.7	18.3
15.	Cybereason	75	95.6	19.4
16.	Fortinet	53	97.2	39.8

For all test I would call Avast the winner followed by Bitdefender and Kaspersky.
Microsoft is severely punished by the performance test.

 

[bayasdev]

MS Defender doesn't stops amazing me, each day it gets closer to the top players in terms of malware detection but they still need to further optimize the engine by implementing some kind of caching like other AVs do.

 

[Vitali Ortzi]

MS Defender doesn't stops amazing me, each day it gets closer to the top players in terms of malware detection but they still need to further optimize the engine by implementing some kind of caching like other AVs do.

it does cache from the ram and physical storage dynamically of signatures and scanned file hashes

 

[SeriousHoax]

Vendors who make products specially for business did far worse than vendors who have both home and business products.
There's hardly any difference between the top products but Real World, Malware Protection, False Positives and Performance tests combined the overall winner here is Avast.
Bitdefender, ESET, Kaspersky in second place.

 

[bayasdev]

Vendors who make products specially for business did far worse than vendors who have both home and business products.

Those vendors often have "FREE" editions of their products to gain more threat intelligence and improve their paid/enterprise products

 

[Vitali Ortzi]

As I like to say, the first system tweak I do on weak computers is to completely disable Microsoft Defender, because it's really hard for the old PC.

What really surprises me is the ever good (there are a lot of false positives, but the protection is quite good, according to the Real-World Protection Test July-October 2020) results of Panda in av-comparatives tests, although in amateur tests it is practically one of the worst known antiviruses.

with all the love i have for windows defender disabling is a must but only on a really low end system

 

[SeriousHoax]

Those vendors often have "FREE" editions of their products to gain more threat intelligence and improve their paid/enterprise products

You're right. I have also said this before. Avast and WD has always been free so, that's that.
Later Bitdefender and then Kaspersky released a free version with the main purpose of gathering threat intelligence.

 

[ForgottenSeer 89360]

Vendors who make products specially for business did far worse than vendors who have both home and business products.
There's hardly any difference between the top products but Real World, Malware Protection, False Positives and Performance tests combined the overall winner here is Avast.
Bitdefender, ESET, Kaspersky in second place.

Glad it’s not only me who sees their great performance

 

[Andy Ful]

Windows Defender is tested by AV-Comparatives in the business environment without most of the advanced settings, which is strange (ASR rules, Network Protection, and CFA are disabled):
"Microsoft: Cloud protection level set to “High”, Cloud-delivered protection set to “Advanced”. Google Chrome extension “Windows Defender Browser Protection” installed and enabled."
https://www.av-comparatives.org/tests/business-security-test-2020-march-june/#product-settings

The advanced settings (ASR rules enabled) are tested by MRG Effitas in 360 Assessment & Certification tests.

 

[SeriousHoax]

Windows Defender is tested by AV-Comparatives in the business environment without most of the advanced settings, which is strange (ASR rules, Network Protection, and CFA are disabled):
"Microsoft: Cloud protection level set to “High”, Cloud-delivered protection set to “Advanced”. Google Chrome extension “Windows Defender Browser Protection” installed and enabled."
https://www.av-comparatives.org/tests/business-security-test-2020-march-june/#product-settings

The advanced settings (ASR rules enabled) are tested by MRG Effitas in 360 Assessment & Certification tests.

These changes were made because Microsoft asked them to do so. It's surprising that they didn't want the ASR rules to be enabled. Something related to false positives maybe?

 

[oldschool]

It's surprising that they didn't want the ASR rules to be enabled. Something related to false positives maybe?

Most likely.

 

[Andy Ful]

These changes were made because Microsoft asked them to do so. It's surprising that they didn't want the ASR rules to be enabled. Something related to false positives maybe?

I think that the setup was adjusted to the test scenario which does not include targeted attacks. The samples were gathered just like in the consumer tests (Home user scenario) and only the AV settings were applied in accordance with the vendor’s guidelines. So, adding the ASR rules, Network protection, etc., would not change the test results and could increase the false positives rate. In MRG Effitas tests, the targeted attacks are included in the Exploit/Fileless part and this makes an important difference.

Edit.
The WD setup from this test is similar to ConfigureDefender HIGH settings (without ASR rules and Network Protection).

 

[ForgottenSeer 89360]

There are products in this test I haven’t even heard of and there are others missing unfortunately. Trend Micro Worry-Free and McAfee ENS/MVISION are not presented.

 

[Andy Ful]

The cumulative statistics (August-November 2018 - August-November 2020)

AV-Comparatives Business Security (sorted by the sum of missed samples)

-- AV Vendor--------------- Total missed samples
Bitdefender.................=6____7
Microsoft....................=13__10
Avast...........................=22___3
Panda..........................=10__18
VIPRE..........................=14__22
Kaspersky...................=9___35
Sophos........................=30__20 (first scoring averaged)
ESET............................=30__34
Fortinet.......................=84__36
CrowdStrike................=81__73
FireEye........................=155_71

The first value in a__b is the total number of missed samples in Real-World tests, the second is the same for Malware Protection tests.

I am not sure how a__b can be combined together in the business environment, but it seems that the missed samples in Malware Protection tests are less important. Also, the errors related to the small number of tested samples and the small number of tests do not allow differentiation between Avast, Bitdefender, Kaspersky, Microsoft, Panda, and Vipre to see the protection in the wild. The scorings of Sophos and Eset are very close to the top AVs. Fortinet, CrowdStrike, and FireEye have got scorings sufficiently low, to differentiate them from other AVs.

Business Security Test 2018 (August - November) - AV-Comparatives (av-comparatives.org)
Business Security Test 2019 (March - June) - AV-Comparatives (av-comparatives.org)
Business Security Test 2019 (August - November) - AV-Comparatives (av-comparatives.org)
Business Security Test 2020 (March - June) - AV-Comparatives (av-comparatives.org)
Business Security Test 2020 (August - November) - AV-Comparatives (av-comparatives.org)