MRG Effitas 360 Assessment Quarter 4

[509322]

Of course you are correct, being a huge malware DB (there are millions of malware variants), who create these tests has to choose a limited number of malware in testing, but statistically representative: new threats and known threats.

But the selection criteria of the various categories are extremely complex.
For example, it is possible to test an antivirus against a higher number of threats, even if in reality they are not very popular and therefore there is little probability that the real user can become infected. Or you can select the threats, which constitute the majority of the causes of infections for common users.

Also by understanding what are the potentially most viral threats may change scenario due to many variables, for example, there are huge differences in the spread of infections on geographical level.
The tests can vary a lot by depending on the parameters chosen by who performs the test, and thus the results may change when you are testing the same malware in different tests.
Frequently it happens that an antivirus program will obtain an excellent result in a test, and mediocre in others.
So these tests may show a statistically correct evaluation of a product, but when the user sees the full green line in the chart with 100% detections, the subliminal message is clear but the reality may be different.

That's why I don't believe in these tests, technically correct, but the evaluation factors are limited to the perimeter of the test.

Well, that's people's fault for not reading the reports carefully in their entirety and doing some investigative report to learn what the test results actually mean - as opposed to what they think it means based upon a few bar charts.

It's human nature at work. 100 % = awesome, I will buy and install that one...

I think these tests are more than just a bit misleading and what you don't know about them can hurt you.

 

[HarborFront]

I don't know why that Norton is specified as Symantec Norton Security. Norton Antivirus and Internet Security were renamed\rebranded Norton Security with different levels like Standard, Deluxe and Premium to differentiate the products.

Those different levels for Norton Security Products have no impact on its core protection (AV + FW + BB + SB). See the link below and you can see that

Norton 2017 Products and Services | Singapore

Coming back to the test. It seems Norton's BB is acting very well

 

[RejZoR]

this version of avast (12) hasn't integrated AVG's behavior blocker yet. Perhaps Behavior block here is evo-gen

avast! did have behavior blocker even in 2016 and back, just different than most others used or have. DeepScreen (now part of CyberCapture), while not overwhelmingly awesome, it does detect malware. Evo-Gen is usually not counted as behavior detection, though it's technically not signature either.

The signatures for Evo-Gen begin life in massive similarity lookup on avast! data servers. They feed unknown received files to the cluster and it then compares them to known malware and known clean software. Then it measures the distance to/from each and if the distance from the cluster of malware is small, it flags it as malicious (same applies to cluster of clean files) and generates an algorithm based signature which can detect many malwares based on that "extract" from the similarity search in the lab.

 

[mal1]

Only 360 samples were used in the test! I'm not a statistician, but is this scientifically accepted? I think the cutoff for the number of samples used in these tests should be..... I don't know, but definitely more than 360!

 

[Winter Soldier]

Only 360 samples were used in the test! I'm not a statistician, but is this scientifically accepted? I think the cutoff for the number of samples used in these tests should be..... I don't know, but definitely more than 360!

I also think so, the statistical validity should be directly proportional to the number of objects.
That's why I think this and other similar tests leave the time they find.

 

[RejZoR]

Only 360 samples were used in the test! I'm not a statistician, but is this scientifically accepted? I think the cutoff for the number of samples used in these tests should be..... I don't know, but definitely more than 360!

It is for real-world tests. And people complained that my videos are too long and they were just half an hour with 100 samples executed in 100% unscientifical way. If you want to do it properly at almost 4x the samples, it takes very long time to perform the test.

 

[Deleted member 178]

Only 360 samples were used in the test! I'm not a statistician, but is this scientifically accepted? I think the cutoff for the number of samples used in these tests should be..... I don't know, but definitely more than 360!

Doesn't matter , which user will ever cross that amount of malware, there is no such things as "real world" in those tests. Those are just statistical results.

You want Real World Tests ? ask them to use UAC and smartscreen, because i never see any installation of Windows without UAC & Smartscreen disabled...then ask them to show how many samples required the users to click "yes" to UAC and SS.
Then once you find enough samples bypassing both SS and smartscreen , you can have a realistic result.

 

[Deleted member 178]

By the way, i have the official answer (from the test lab) that UAC and Smartscreen were disabled on this test.

So as i said earlier, this test has nothing to do with real world use. Take it a a statistic info.