AV-Comparative Real-World Protection Test February-June 2017

5

509322

Thread author
AV-C has minimum requirements for testing. Those minimum requirements ensure that all the softs tested are decent.
 

VeeekTor

Level 5
Verified
May 16, 2017
197
Emsisoft seems to have several of these same tests results, go back several months and you'll always notice the ghetto results.
 
5

509322

Thread author
Emsisoft seems to have several of these same tests results, go back several months and you'll always notice the ghetto results.

"Ghetto results" ?

What ? That a user might have to respond to a behavior blocker alert ?

The product can be made fully automated if the user so desires (set auto-quarantine for the behavior blocker) by changing only a single setting. If AV-C made that single setting change and tested, then the bar graphs in each AV-C test would be virtually all green with a tiny sliver of red for Emsisoft.
 

ttto

Level 9
Verified
Well-known
Sep 22, 2016
408
Good results from BD and Kaspersky, Trend Micro too, but too much false positives. That Panda result sounds very suspicious for me.
 

VeeekTor

Level 5
Verified
May 16, 2017
197
"Ghetto results" ?

What ? That a user might have to respond to a behavior blocker alert ?

The product can be made fully automated if the user so desires (set auto-quarantine for the behavior blocker) by changing only a single setting. If AV-C made that single setting change and tested, then the bar graphs in each AV-C test would be virtually all green with a tiny sliver of red for Emsisoft.

Check the results from the last 4 or 5 tests...Always on its back looking up...
 

VeeekTor

Level 5
Verified
May 16, 2017
197
"Ghetto results" ?

What ? That a user might have to respond to a behavior blocker alert ?

The product can be made fully automated if the user so desires (set auto-quarantine for the behavior blocker) by changing only a single setting. If AV-C made that single setting change and tested, then the bar graphs in each AV-C test would be virtually all green with a tiny sliver of red for Emsisoft.

You must work for semisoft products?
 
5

509322

Thread author
Check the results from the last 4 or 5 tests...Always on its back looking up...

You must not have read the test methodology. AV-C rates a user decision a pass. The yellow portion of the bar graphs are not the product failing to protect the system.

Like I said, all a user has to do is set the behavior blocker to auto-quarantine and that will completely eliminate user decisions. So, to your way of thinking - which is obviously that a product is a fail if the bar graph is not entirely green. By changing the behavior blocker to auto-quarantine the entire bar graph for Emsisoft would be green with a tiny sliver of red.

Also, the test which is referenced in this thread includes data from all previous tests. Based upon that cumulative test data the overall actual compromise rate of Emsisoft = 0.15 %. That's a better result than Kaspersky, ESET, all the others except Bitdefender and a tie with F-Secure if I recall the figures correctly. Within the context of typical computing, a compromise rate of only 0.15 % means that Emsisoft users are extremely well protected.

You must work for semisoft products?

No. I just know how to interpret data instead of simply looking at bar graphs. There is no Emsisoft conspiracy on the forums.

Everybody that has read this thread knows you have something against Emsisoft and your ignorance of test methodology and test data shows.

You are not referencing any specific data that definitively proves your assertions.

In fact, all the AV-C test data contradicts what you are trying so hard to convince others to believe.
 
Last edited by a moderator:

VeeekTor

Level 5
Verified
May 16, 2017
197
You must not have read the test methodology. AV-C rates a user decision a pass. The yellow portion of the bar graphs are not the product failing to protect the system.

Like I said, all a user has to do is set the behavior blocker to auto-quarantine and that will completely eliminate user decisions. So, to your way of thinking - which is obviously that a product is a fail if the bar graph is not entirely green. By changing the behavior blocker to auto-quarantine the entire bar graph for Emsisoft would be green with a tiny sliver of red.

Also, the test which is referenced in this thread includes data from all previous tests. Based upon that cumulative test data the overall actual compromise rate of Emsisoft = 0.15 %. That's a better result than Kaspersky, ESET, all the others except Bitdefender and a tie with F-Secure if I recall the figures correctly. Within the context of typical computing, a compromise rate of only 0.15 % means that Emsisoft users are extremely well protected.



No. I just know how to interpret data instead of simply looking at bar graphs. There is no Emsisoft conspiracy on the forums.

Everybody that has read this thread knows you have something against Emsisoft and your ignorance of test methodology and test data shows.

You are not referencing any specific data that definitively proves your assertions.

In fact, all the AV-C test data contradicts what you are trying so hard to convince others to believe.


Now, now, let's not attack the messenger, and let's not be so defensive, and let's not try to substantiate your position, by saying "Everyone knows you have something against Emsisoft", because that is simply not true.

I have purchased Emsisoft, and will confess it was less than stellar... So no more defensive drama. OK.
 
5

509322

Thread author
I have purchased Emsisoft, and will confess it was less than stellar...

L0L, now I understand your basis for judgment. "Less than stellar..." Each user has their own unique personal experience with software. So statistics alone are not the entire picture.

Not every single user is going to have an experience that jives with paper results and other descriptions.

You didn't mention your experience earlier - or I just missed it.

I mistakenly assumed your point was all about the statistics in the test report.
 
Last edited by a moderator:

VeeekTor

Level 5
Verified
May 16, 2017
197
L0L, now I understand your basis for judgment. "Less than stellar..." Each user has their own unique personal experience with software. So statistics alone are not the entire picture.

Not every single user is going to have an experience that jives with paper results and other descriptions.

You didn't mention your experience earlier - or I just missed it.

I mistakenly assumed your point was all about the statistics in the test report.




If you can read minds, then my statement "less than stellar" is open to assessment.

But over all as per the stated test results, for the last few months and my own use of the product I would say it would not be my first, second, or even third choice of a security product for daily use.
 

insanity

Level 5
Verified
Oct 9, 2016
216
Based upon the data, the true compromise rate of Emsisoft is:

(3/1955) x 100 = 0.15 %

Honestly, I don't see your metric for compromise rate as accurate. I understand the reason why the Labs award a low score to Emsisoft. A user-required action is not a block. Sometimes I see posts on the Malware Hub and it seems that the testers count an user-dependent window as a block (which implied that the system was protected). I can't agree with that! You cannot expect the user to always block the threat. If you're testing a security app and a security alert shows up, of course you're going to block because you know those files are malware. But in a real life scenario, the user might not have enough information to decide for the block. And here is the problem for Emsisoft: it asks too much of the user decision, more than I would deem appropriate. And bear in mind that these results don't include the user-required action when dealing with legitimate programs. Once I remember installing some programs, and all the time Emsisoft would display a Behaviour Alert for a certain registry key, DLL etc. So now I disable Emsisoft when I install some new programs, and I manually whitelist certain pieces of software before running them. This is not ideal, since an AV is not supposed to be a Default-Deny solution.
So, what happens is that if an antivirus displays too many alerts, the user may start to question the effectiviness of the AV. And if the user doesn't trust the AV, it might eventually ignore an alert when a real infection threatens the system. It's about the user (human) behaviour.
A user-required action is not a block, but shouldn't be counted as a miss, either. I think the best approach should be award scores for threats that have been blocked, missed and user-dependent as well as false positives.
 

GonzitoVir

Level 5
Verified
Well-known
May 16, 2017
198
A user-required action is not a block... You cannot expect the user to always block the threat.
Agree. The majority of people doesn't know what those alerts mean and may click the "Allow" (malware) button, instead of "Block".
My 12 years old nephew and his friends never read those popups or even the messages when installing a new game or other app, so they just click on the Next buttons and install all extra software (PUPs) in the process.
 
  • Like
Reactions: shukla44
5

509322

Thread author
My response here is not meant to be specific to Emsisoft only, but instead a reply to the general attitude regarding security software at-large - and these lab test reports.

Honestly, I don't see your metric for compromise rate as accurate.

The metric is exactly as AV-C reported it based upon the cumulative testing it performed. I didn't make it up out of nowhere. 3 compromises out of 1955 files. Read the AV-C test methodology.

A user-required action is not a block. Sometimes I see posts on the Malware Hub and it seems that the testers count an user-dependent window as a block (which implied that the system was protected). I can't agree with that! You cannot expect the user to always block the threat.

The test lab does not consider user decisions to be a fail. That's a generally accepted industry practice.

When an alert appears, and the file is prevented from proceeding any further, the system is protected. When in doubt, use the recommended action and quarantine. Nothing is permanently broken.

But in a real life scenario, the user might not have enough information to decide for the block.

When in doubt block and then investigate. This is not difficult and it isn't unrealistic to expect a user to do so. "Better safe than sorry" habits are good habits. In time, a user learns the product and becomes more self-reliant.

The vast majority of users don't avail themselves of the IT security learning resources that security soft publishers make available. And whose fault is that - the publisher's ?

A user should not automatically have doubts that a file downloaded from a website with a good reputation is suspect when a behavior blocker alert appears. That's like constantly worrying that Windows Updates might be malicious. What the real problem is that some users expect a security soft to inspect and tell them every single file is safe - without the user bearing any responsibility in using common sense within the context of what they are doing on the system at the time an alert appears.

This is not ideal, since an AV is not supposed to be a Default-Deny solution.

A behavior blocker alert is not default-deny. A legitimately safe file triggering a behavior blocker alert does not make it default-deny. If you know that a bunch of softs that you use trigger behavior blocker alerts, then you can get them whitelisted and\or make exclusions for them in the product. This is not difficult.

So, what happens is that if an antivirus displays too many alerts, the user may start to question the effectiviness of the AV.

I have personal experience with Emsisoft. I use quite a bit of utilities that generate behavior blocker alerts. Within the context of what I am doing on the system and what is triggering the behavior blocker, it is fairly obvious. I mean it should be common sense for anyone who has used security softs for a while. For example, I run a Dell driver update utility. Every time it is run, it triggers an AMN query and behavior blocker alert. It's common sense that it is safe to always Allow. And if the alert is expanded, the AMN Safe rating is clearly shown. So there is no great mystery nor burden required of the user.

The type of users to which you seem to be referring are those people that are so oblivious and ignorant of IT matters that no matter what security soft is installed there are sure to be issues. The industry cannot do anything about those type users. And the industry cannot fool-proof software. With the current state of technology, the user is expected - and really has no choice - but to learn as they go. Until Skynet comes along, this fact is not going to change. You have to remember, organizations with billions upon billions of dollars at their disposal who have poured billions and billions of dollars into improving IT security have been unable to accomplish what some users want and expect with the current state of technology. If it were that easy, then the industry would have accomplished it long ago.

It seems to me that there is only a small minority of people that complain, because what they unrealistically want and expect is a solution that is fully automatic with 100 % detection under all circumstances, never require any user decision-making, 0.1 % system resource usage, and 100 % compatibility. It doesn't work that way. It is the same old complaints that have been around forever - some users automatically blame a security soft because they don't know what they're doing instead of putting a little bit of effort into actually paying attention and attempting to learn.

False positives are blown out of proportion unless they get out of hand. Out of approximately 2000 files, a false positive rate of 1.5 % or less is reasonable.
 
5

509322

Thread author
Agree. The majority of people doesn't know what those alerts mean and may click the "Allow" (malware) button, instead of "Block".
My 12 years old nephew and his friends never read those popups or even the messages when installing a new game or other app, so they just click on the Next buttons and install all extra software (PUPs) in the process.

It is the parents' responsibility. Children that don't know any better should be locked out of the system. There is no way you can pass responsibility onto the security software publisher for a minor's actions on a computer.

At least in the United States there are legal precedents for this and I'd bet it is the same in the EU and other countries.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top