MRG Effitas MRG Effitas 360 Assessment & certification Q3 2019

[Andy Ful]

The full report:

https://www.mrg-effitas.com/wp-content/uploads/2019/11/MRG_Effitas_2019Q3_360.pdf

Important charts for the home users

One test means nothing, so I made a short report that includes the data from all three quarters in the year 2019.
Two categories were skipped: Ransomware and Exploit/Fileless. In the first category, all AVs blocked all Ransomware samples in the year 2019. The second is not relevant for the home users, because it was performed with highly vulnerable & unpatched (for years) software (this test can be relevant for enterprises and organizations).

Missed on first detection (number of samples: malware 1053, financial 265, PUA/Adware 25)
Symantec............0/1053 + 0/265 + 1/25.............= 1
Kaspersky...........0/1053 + 0/265 + 2/25.............= 2
Eset....................1/1053 + 0/265 + 2/25..............= 3
BitDefender.........1/1053 + 0/265 + 6/25.............= 7
Avira....................3/1053 + 1/265 + 3/25.............= 7
CrowdStrike.......7/1053 + 1/265 + 5/25..............= 13
Defender.............12/1053 + 0/265 + 1/25...........= 13
F-Secure.............10/1053 + 2/265 + 8/25............= 20
Avast..................26/1053 + 20/265 + 1/25......... = 47
Trend Micro.......46/1053 + 17/265 + 7/25..........= 70

Still missed after 24h
Symantec........0/1053 + 0/265 + 1/25...............= 1
Kaspersky........0/1053 + 0/265 + 1/25..............= 1
Eset.................1/1053 + 0/265 + 1/25=.............= 2
F-Secure..........2/1053 + 0/265 + 1/25...............= 3
BitDefender.....1/1053 + 0/265 + 6/25..............= 7
Defender.........3/1053 + 0/265 + 0/25.............. = 3
Avira................3/1053 + 1/265 + 3/25.............. = 7
CrowdStrike....4/1053 + 0/265 + 4/25...............= 8
Trend Micro....17/1053 + 0/265 + 2/25.............= 19
Avast...............17/1053 + 14/265 + 1/25...........= 32


The best: Eset, Kaspersky, Symantec, BitDefender. The last has slightly too much missed PUA/Adware samples.

The good: Avira, CrowdStrike, Defender, F-Secure

The rest: Avast, TrendMicro

In the final scoring, the PUA/Adware was treated as less important than malware and financial samples.

 

[Burrito]

The full report:

https://www.mrg-effitas.com/wp-content/uploads/2019/11/MRG_Effitas_2019Q3_360.pdf

The best: Eset, Kaspersky, Symantec, BitDefender. The last has slightly too much missed PUA/Adware samples.

The good: Avira, CrowdStrike, Defender, F-Secure

The rest: Avast, TrendMicro

In the final scoring, the PUA/Adware was treated as less important than malware and financial samples.

Thanks Andy.

Concur with PUPs/PUAs being given less weight. While it's good for an AV to detect them, there are other ways, other tools... maybe not the AVs most important task.

PUA Misses

The Exploit/fileless samples test (not shown in OP) was also interesting, giving Norton & Kaspersky rare misses.

Crowdstrike is like Cylance. They are good at what they do. They are just best not used stand-alone.

 

[Andy Ful]

Defender and the Exploit/Fileless test.
Unfortunately, MRG Effitas uses Windows 7 and Windows 10 in this test, even when in the methodology section only Windows 7 is mentioned. There is no way to know from the report, how many samples require Windows 10. Of course, the samples which were run on Windows 7 cannot be stoped by Defender, because it is not an antivirus (only anti-spyware engine is present). In the Exploit/Fileless test, Windows 7 is used because it is a common Windows version in organizations and enterprises. As we can see from several reports, the operating systems/software in organizations and enterprises are vulnerable to exploits which were patched years ago (like Eternal Blue used in WannaCry).
Anyway, in MRG Effitas 360 Degree Assessment & Certification tests the AV premium versions are tested (except Defender), which often have anti-exploit modules, so they can protect better than Defender on Windows 10 with default settings. In the Exploit/Fileless test, Defender could compete with premium AVs only with activated advanced settings, especially ASR rules.

 

[ForgottenSeer 823865]

@Andy Ful sum it all, this is one if the reason i despise test labs, flawed or biased methodologies, just marketing proxies for vendors.

About WD, everyone knows, it won't do well against fileless malware, by default it is just a RT scanner, to be a valid test , Win10 enterprise should be used with the ATP and Exploit Protection feature.

 

[SeriousHoax]

Nice, ESET did really well in this Q3 2019 test. Only product to score 100% in all category.

 

[Andy Ful]

Ha, ha. It is always nice if your favorite horse wins 3 races in a single quarter of the year, even if it is a pure coincidence. The opposite situation is not nice, even if you know that it is probably a coincidence, too.
That is why some smart people invented statistics to make people less happy and less unhappy.

 

[SeriousHoax]

Ha, ha. It is always nice if your favorite horse wins 3 races in a single quarter of the year, even if it is a pure coincidence. The opposite situation is not nice, even if you know that it is probably a coincidence, too.
That is why some smart people invented statistics to make people less happy and less unhappy.

I know I know. I'm just saying ESET did a good job in this test. I'm not shouting, "East or West, ESET is the best"

 

[Andy Ful]

I know I know. I'm just saying ESET did a good job in this test. I'm not shouting, "East or West, ESET is the best"

I was proud when Poland played very well in football Euro 2016, even if I knew that by coincidence most Polish players were in perfect shape again after 40 years, and this could not last long.
By the way, ESET scores very well in MRG Effitas tests.