Andy Ful

Level 48
Verified
Trusted
Content Creator
Te latest AV-Comparatives Real-World Protection Test (Jul-Aug 2019):

Test chart:
https://www.av-comparatives.org/com...chart_month=Jul-Aug&chart_sort=1&chart_zoom=2

Test document in PDF:
Real-World Protection Test Jul-Aug 2019 – Factsheet | AV-Comparatives

***********************************************************************************
False positives

The false-alarm test in the Whole-Product Dynamic “Real-World” Protection Test consists of two parts: wrongly blocked domains (while browsing) and wrongly blocked files (while downloading/installing).
https://www.av-comparatives.org/real-world-protection-test-methodology/
***********************************************************************************

From AV-Comparatives Real-World test, one cannot conclude how many false positives can produce the AV engine while downloading/executing the files.
This is tested separately for AV-Comparatives Malware Protection tests in False Alarm tests, for example: False Alarm Test March 2019 | AV-Comparatives
AV-C false positicves III 2019.png

Current Malware Protection test and False Alarm test will be probably published soon.
 
Last edited:

SeriousHoax

Level 10
Verified
Malware Tester
The results are based on the test set of 352 live test cases (malicious URLs found in the field), consisting of working exploits (i.e. drive-by downloads) and URLs pointing directly to malware.
What does it mean actually? AVs that fail to block the malicious url but blocked the sample locally is considered a pass too, right?
 

shmu26

Level 83
Verified
Trusted
Content Creator
From AV-Comparatives Real-World test, one cannot conclude how many false positives can produce the AV engine while downloading/executing the files.
This is because the file FPs are grouped together with the url FPs, correct?
I looked at the False Alarm Test from March that you linked us to, and I see that Windows Defender is the "few FPs" category, along with Bitdefender and Kaspersky.
 

Andy Ful

Level 48
Verified
Trusted
Content Creator
Correct Analysis
1- BitDefender 100% 0 False positive
2-Kaspersky 99.1% 0 False Positive
3-McAfee 99.7% 3 False Positive (More False Positive)
4-Symantec 100% 4 False Positive ( More more False positive)
5-Windows Defender More More More More False Positive
You cannot say that your analysis is correct, without presenting the correct/accepted criteria.
If not, then anyone could post "his correct analysis" which would be very different from yours.
Your criteria do not rely on data (from this test) based on the number of blocked samples and the number of false positives (where is Avira 100% with 1 false positive, or K7, Total Defense, Vipre 99.7 with 1 false positive?).:emoji_thinking:
 
Last edited:

mlnevese

Level 16
Verified
The URL false positives are usually less troublesome, especially those with low prevalence.(y)
The only problem is when the URL false positive stops you from doing your work. Kaspersky used to report many government sites I need for my work as "potential data loss". They eventually solved it but for a time I had to whitelist those sites and some would get blocked even when whitelisted.
 

blackice

Level 11
Verified
The URL false positives are usually less troublesome, especially those with low prevalence.(y)
I always push back on the railing on M$ for false positives. Anecdotally it’s always people who dabble in esoteric software that most users would never even know existed. Even a lot of advanced users aren’t using tons of tools with low prevalence, which is where most FPs come from.

Now your discussion of how FPs are done in the testing and contrasting URL vs Files really shines a light on how misguided this “concern” is for most users.
 

conceptualclarity

Level 21
Verified
Trusted
Content Creator
False positives are highly over-rated as a negative.
I disagree. If a security product does a lot of false positives I find it a strong disincentive to scan with it, because i think "I just don't have time now to deal with all those false positives." It's especially frustrating when a false-positive-prone program shows me registry keys with only numbers, nothing to indicate what the keys actually have to do with.

But as for an anti-virus product, I flat out am not going to use it if makes a habit of going after harmless things on my computer.
 
9

93803123

I disagree. If a security product does a lot of false positives I find it a strong disincentive to scan with it, because i think "I just don't have time now to deal with all those false positives." It's especially frustrating when a false-positive-prone program shows me registry keys with only numbers, nothing to indicate what the keys actually have to do with.

But as for an anti-virus product, I flat out am not going to use it if makes a habit of going after harmless things on my computer.
You shouldn't need to run a scan more than once. And that's immediately after installing it on a known, clean-state system. Afterwards, you are going to have to have faith in your choice that the real-time protection will protect you. Scanning a system more than once per month is a waste of your time and your system resources.

Very few users have major issues with false positives. A small percentage have annoyances with false positives or, more likely, the publisher rates a program as PUA\PUP.