AV-Comparatives AV-Comparatives Real-World protection July-August 2019

[Cortex]

The URL false positives are usually less troublesome, especially those with low prevalence.

As I said it tends to be those who know little about security whose PC's I maintain, & or relatives (though I'm trying to move away from doing this) who have problems with FP's, to them its a malicious site or malware, they don't have the knowledge or experience to differentiate between the two - I don't have an issue with this.
Edited to make some sense ~

 

[TairikuOkami]

It's especially frustrating

Ditto. When I test AV or clean install Windows, WD removes many of my files and customized shortcuts, it is extremely annoying.

But as for an anti-virus product, I flat out am not going to use it if makes a habit of going after harmless things on my computer.

Exactly, like when I install Kaspersky and try to check processes via Process Hacker (not running as admin) and whoops, it is gone.

 

[mlnevese]

False positives is not a problem to most enthusiasts that frequent forums like this. If it happens they know how to deal with it. BUT most people are not enthusiasts. In a situation like mine a lawyer in my country who was using Kaspersky back when it blocked government sites would be unable to access any of the internal Court systems and would be unable to work with that computer. Most people don't have any idea what black list and white lists are or how to manage them.

In other words a consumer level product must have as few FP as possible. It also should be as automatic as possible. If it's configurable so that a more experienced user can have a higher degree of control then it's good for both the experienced and the inexperienced user.

A business protection product on the other hand is meant to be configured by a specialist and should give as much control to that professional as possible.

 

[Andy Ful]

In a situation like mine a lawyer in my country who was using Kaspersky back when it blocked government sites would be unable to access any of the internal Court systems and would be unable to work with that computer. Most people don't have any idea what black list and white lists are or how to manage them.
...

This situation is very rare. Usually, AVs blocks URLs (false positives) when the website hosted something malicious (hacked website in the past) or hosted something that was wrongly recognized as malicious. I am curious how Windows Defender will score in the upcoming False Alarm test. The total number of false positives (URLs + files) in the current real-world test is rather high, but there is no information about samples' prevalence.

 

[The Rectifier]

Real-World Protection Test Jul-Aug 2019 – Factsheet | AV-Comparatives

I am particualry worried about ESET's performance in the latest AV-Comparatives. It consistenly falls behind the other AVs.

 

[jetman]

Symantec/Norton with a strong performance again.

I'd like to see what difference it would make if they tested Kaspersky with the Trusted Application Mode enabled. This is a feature that most applications do not offer.

 

[93803123]

I am particualry worried about ESET's performance in the latest AV-Comparatives. It consistenly falls behind the other AVs.

Please explain what the practical difference between 0.8 and 1.7 % are ? That is 3/352 and 6/352 test cases. What is there to worry about between those two values ?

See. Just what I said. People measure by the observed failures, and not the successes. ESET was successful in 346 out of 352 test cases.

 

[The Rectifier]

Please explain what the practical difference between 0.8 and 1.7 % are ? That is 3/352 and 6/352 test cases. What is there to worry about between those two values ?

See. Just what I said. People measure by the observed failures, and not the successes. ESET was successful in 346 out of 352 test cases.

I see your point. You are right the difference is very small. The thing that bothers me is not the value of the difference. It is the consistency that this difference occurs. Other vendors seem to maintain a 0% compromised in all these tests for many months. For example Bitdefender and Microsoft maintain a 0% compromised since February. It has been a while since ESET has scored scored a 0% compromised in a real world protection

 

[roger_m]

I see your point. You are right the difference is very small. The thing that bothers me is not the value of the difference. It is the consistency that this difference occurs. Other vendors seem to maintain a 0% compromised in all these tests for many months. For example Bitdefender and Microsoft maintain a 0% compromised since February. It has been a while since ESET has scored scored a 0% compromised in a real world protection

ESET has excellent signatures and is almost always one of the first antiviruses to add signatures for new threats. With its default settings, the proactive protection could be better, but this can be tweaked to work better.

Advice Request - Configure ESET Antivirus for Maximum Security (by RoboMan)

Last update: November 2021 If you're here you may probably have been delighted already by the majestic features of ESET :) Maybe the signatures convinced you? Great static detection for sure. In this thread I will guide you a bit on how to configure your ESET product for maximum security...

malwaretips.com

With Robbie's custom settings, it would score better.

 

[Nightwalker]

ESET has excellent signatures and is almost always one of the first antiviruses to add signatures for new threats. With its default settings, the proactive protection could be better, but this can be tweaked to work better.

Advice Request - Configure ESET Antivirus for Maximum Security (by RoboMan)

Last update: November 2021 If you're here you may probably have been delighted already by the majestic features of ESET :) Maybe the signatures convinced you? Great static detection for sure. In this thread I will guide you a bit on how to configure your ESET product for maximum security...

malwaretips.com

With Robbie's custom settings, it would score better.

Those settings arent that good and wont change too much, if one wants to tweak ESET, it is much better to use those HIPS rules crafted by its staff:

Advice Request - Eset Internet Security v11 - Best Tweaks?

I just installed EIS v11 on my PC based on the many positive recommendations in this forum. I have HIPS set to Smart Filtering, and the Firewall set to Interactive. Both settings I understand, and can deal with well. That said, are there any other tweaks that I should make that would not make...

malwaretips.com

[KB6119] Configure HIPS rules for ESET business products to protect against ransomware

support.eset.com

About ESET signatures, yes it has good signatures, but what is really great is its advanced heuristics (a form of zero day protection), most zero day malware are detected by ESET using it and some people think erroneous think that it was a signature detection.

I think there is no product with better heuristics and generic family malware detection than ESET (DNA Detection), it has always been its strong point after all.

 

[93803123]

I see your point. You are right the difference is very small. The thing that bothers me is not the value of the difference. It is the consistency that this difference occurs. Other vendors seem to maintain a 0% compromised in all these tests for many months. For example Bitdefender and Microsoft maintain a 0% compromised since February. It has been a while since ESET has scored scored a 0% compromised in a real world protection

I understand. A consistent record of 0 % compromise can be reassuring. Yet I wouldn't put a whole lot of faith in that kind of consistency.

Perhaps ESET is being honest while Bitdefender and Microsoft are gaming the test ? Or it could be any of a myriad of circumstances that affect these av lab test outcomes, yet are never revealed to the report readers.

 

[93803123]

Well also while some vendors get 100% on mainstream published AV tests, nobody aced the malware sample packs in the malware hub.

0% in those tests clearly doesn’t mean you’re safe from compromise, so as others have said, test results have to be taken with a grain of salt.

Consistent 0 % compromise result in a 350 sample av lab test cannot be linearly extrapolated to 0 % clean in a 1000 sample test or 2000 or 4000 or 8000 and so on. Statistics and reality do not work that way, but people make that extrapolation in their minds every time they look at such tests. As the sample size, or system usage increases, the probability of a compromise increases. The shape of that compromise probability curve is not linear or smooth. It is complex, increasing and decreasing over time.

If one is an indiscriminate downloader, then a PUP or PUA infection is almost certain. A more serious infection, it is difficult to quantify the probability, but it is less.

 

[mlnevese]

My personal experience is that I haven't seem a single malware in my online activity for years. Saw some scam sites though. Otherwise the only recent live malware I saw was in an infected pendrive one of my clients brought to my office. It had a backdoor of some kind and was blocked and erased by Kaspersky

 

[93803123]

My personal experience is that I haven't seem a single malware in my online activity for years. Saw some scam sites though. Otherwise the only recent live malware I saw was in an infected pendrive one of my clients brought to my office. It had a backdoor of some kind and was blocked and erased by Kaspersky

The incidence of serious infection is low.

One should be confident in their choices if one has taken sound steps to protect their local system. There's no need to obsess about an infection.

 

[mlnevese]

I think being a “prolific downloader” is the main way a home user encounters malware. In my recent surveys, basically 100% of the dozens of pirated software I downloaded contained legitimate malware. Unfortunately unlike the “old days” it no longer is blatantly obvious when what you downloaded is bogus. These days they will either bury the real software beneath layers of fake setup.exe files, or worse, I saw a Windows Server installer ISO where the Setup.exe was repacked to drop some malware before unpacking and running the real installer.
Long story short the most likely way you’ll get malware these days as a home user is by partaking in downloading questionable stuff online. Drive by exploits do exist but they’re not at all common.
I can imagine in a business environment, phishing emails and people bringing malware onto the network is far more prevalent.

That's my point. If you stay legal the chance of seeing malware is minimal. If you don't navigate through the dark side of the web the chance of infection is minimal. The problem is when the user downloads every pirated software they can find and don't even check if the video they just downloaded is actually a video...

 

[notabot]

I think being a “prolific downloader” is the main way a home user encounters malware. In my recent surveys, basically 100% of the dozens of pirated software I downloaded contained legitimate malware. Unfortunately unlike the “old days” it no longer is blatantly obvious when what you downloaded is bogus. These days they will either bury the real software beneath layers of fake setup.exe files, or worse, I saw a Windows Server installer ISO where the Setup.exe was repacked to drop some malware before unpacking and running the real installer.
Long story short the most likely way you’ll get malware these days as a home user is by partaking in downloading questionable stuff online. Drive by exploits do exist but they’re not at all common.
I can imagine in a business environment, phishing emails and people bringing malware onto the network is far more prevalent.

Just to say - Recently I opened and scanned a laptop from my student years (~15 yrs old). Back in the day I did download from torrents but only from reputable torrent creators, from memory tpb had a special color for their icons. I was using 1-2 AVs as a student (one was just static) and they never reported anything. I had forgotten about the files but using a modern AV flagged *most* of them with malware.
15 years of progress in signatures and ML static detection revealed that yesterday’s “reputable” torrent creators were in fact seeding malware and getting a clear pass from 2 AVs wasn’t sufficient.
Being the paranoid that I am I was running these from within a VM but the moral is clear pirated stuff is extremely dangerous and I found it worrying that at the time this was the AAA stuff that was the “golden standard”.

Only use software you’ve bought, if it’s too expensive use a big & reputable open source alternative

 

[Venustus]

legitimate malware

I like that

 

[The Rectifier]

I’m in full agreement. But I think part of the appeal of having a suite of fancy anti malware tools is for those who choose to (or have to) handle high risk content, how do we keep ourselves safe?

I will criticize 95% of AV software for having a confusing stand on piracy tools. Most of them will flag things like AutoKMS and other activators and even keygens as PUA or HackTools. But at the same time I’ve seen some (esp BitDefender and Windows Defender) incorrectly flag them as trojans or malware. To make matters worse I’ve also found a fake KMS activator (which just phones home to a botnet domain and downloads a script to run) marked as “just” a KMS tool. In the end I don’t have good advice in terms of running AV scanners on pirated software as the mechanism for determining if it is legitimate. Hopefully anyone playing that game possesses the tools and background knowledge to analyze their own binaries.

EDIT: would also like to point out that BB’s aren’t a magic bullet here either. A lot of piracy tools do things that inherently look suspicious — temporarily installing root certificates, setting themselves to run in early boot, patching binaries on the system and disabling Windows integrity checking, etc. Most BB’s rightfully mark these actions as suspicious but that’s not helpful if you’re trying to determine whether it’s acting as a piracy tool or root kit.

That's exactly what I was experiencing! I solved my problem by using ESET. It has excellent categorization regarding cracking tools etc and in my experience it flags only the truly malicious ones.

 

[Nevi]

View attachment 225232

Real-World Protection Test Jul-Aug 2019 – Factsheet | AV-Comparatives

I am particualry worried about ESET's performance in the latest AV-Comparatives. It consistenly falls behind the other AVs.

I dont think you need to worry. The difference is a few %. Eset is as solid as it has ever been.

 

[Andy Ful]

...
False positives
...

Current Malware Protection test and False Alarm test will be probably published soon.

Here are the results of False Alarm test from September 2019 report:

The results do not take account of sample prevalence. For example, Trend Micro had twice more false positives than Avast. But from the test, it follows that there will be ten times (or more) people who will encounter the false positives with Avast, as compared to Trend Micro.