AV-Comparatives Anti-Phishing Protection in AVs and Browsers – February 2023 - commissioned by Gen Digital

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Gandalf_The_Grey

Level 76
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Apr 24, 2016
6,506
Introduction
This test evaluates the phishing-page detection rates and false positive rates at time of testing (February 2023) of various AV products, and different browsers, as listed below. AV-Comparatives selected the products to be tested. We purchased these in January 2023. This comparative test was commissioned by Gen Digital. The testing dates were chosen by the testing lab and not pre-disclosed.
Tested products
The test took place from 1st February till 19th February 2023. The following up-to-date products were tested in parallel, and with active Internet/cloud-access, on Windows 10. Phishing protection provided by AV products were tested using Google Chrome (with Google SafeBrowsing disabled). The browser extensions of the AV products were installed and enabled. The different browsers were tested without an AV program running.

Antivirus (AV) Products:
• Avast Free Antivirus 22.12 – 23.1
• Avira Free Security 1.1
• Bitdefender Internet Security 26.0
• ESET Internet Security 16.0
• Kaspersky Standard 21.8
• Malwarebytes Premium 4.5
• McAfee Total Protection 16.0
• Microsoft Defender 4.18 with Defender browser plugin for Chrome
• Norton 360 Deluxe 22.23
• Trend Micro Internet Security 17.7

Browsers:
• Avast Secure Browser 109.0 – 110.0
• Google Chrome 109.0 – 110.0 with SafeBrowsing
• Microsoft Edge 109.0 – 110.0
• Mozilla Firefox 109.0 – 110.0
• Opera 95.0
Anti-Phishing Test-Set
For this first test, we used 250 valid phishing URLs. The number of used clean URLs for false alarm detection was 250.
Test Results
1682356002019.png 1682356043866.png
PDF:
 

Trident

Level 27
Verified
Top Poster
Well-known
Feb 7, 2023
1,629
Look at that Mcafee score!!! they have much improved
McAfee Web Advisor was always above average, now has it been better than Kaspersky is hard to tell. But it was one of the first technology, if not the first one to block and rate malicious websites.

What’s going on at Norton, what is this poor score? Compromised 0.4% on Real World protection, second lowest score on the phishing test… May have something to do with rating new, low reputation sites in yellow and not blocking them completely.
 

Trident

Level 27
Verified
Top Poster
Well-known
Feb 7, 2023
1,629
@Trident Indeed,

Norton does not make any sense. Norton should get URL and sample feeds from Avast, Avira and AVG and business solution Symantec Endpoint Protection. In theory it should have the largest / broadest malware intelligence network.

???
I don’t know what they are doing, but since detection varies (like on Real World Protection Norton and Avira had 0.4% misses and Avast had none), as well as looking at this score where Avast emerges as second best and Norton as second worst, it looks like Norton has not integrated any threat feeds whatsoever. I don’t know why this is and for what they sprinkled 360 million + 8 billion.
 

Trident

Level 27
Verified
Top Poster
Well-known
Feb 7, 2023
1,629
Another result surprises me: can someone explain why Firefox scored better than Google when it uses the '30 minute update but private' version of Google safe browsing?
View attachment 274876
Though it uses Google Safe Browsing, there is no guarantee it doesn’t use any heuristics to detect brand impersonation or any URL-reputation logics. This is where the difference may be coming from.
 
F

ForgottenSeer 97327

Though it uses Google Safe Browsing, there is no guarantee it doesn’t use any heuristics to detect brand impersonation or any URL-reputation logics. This is where the difference may be coming from.
Other browsers also have typo protection. Considering the reach and the experience both Microsoft and Google have in AI,webcrawling and the combination of having a browser plus a search engine, it is very unlikely Mozilla would have been able out smarten them.

The only reason I can think of is that Firefox had blocking of HTTP websites enabled (80% of phishing and scam URL's are unecrypted).
 

Trident

Level 27
Verified
Top Poster
Well-known
Feb 7, 2023
1,629
Other browsers also have typo protection. Considering the reach and the experience both Microsoft and Google have in AI,webcrawling and the combination of having a browser plus a search engine, it is very unlikely Mozilla would have out smartened them.

The only reason I can think of is that Firefox had blocking of HTTP websites enabled (80% of phishing and scam URL's are unecrypted).
That may be one reason, also (I’ve not installed Chrome in years so I may be wrong) but by default it gets installed with the enhanced security turned off, doesn’t it? As far as I remember, this relies only on a local copy of the safe browsing cache and doesn’t check every URL in real time.

I am inclined to believe this score as on all my tests Chrome has always performed very poorly against phishing and malicious sites, but haven’t ever tested Firefox so not sure if it’s better. For me this browser is non-existent.
 
  • Like
Reactions: SeriousHoax

SeriousHoax

Level 47
Well-known
Mar 16, 2019
3,630
The only reason I can think of is that Firefox had blocking of HTTP websites enabled (80% of phishing and scam URL's are unecrypted).
I'm surprised by the difference between Chrome and Firefox also but this is not the first time. I have seen massive difference like this in some other professional tests also. Firefox by default auto upgrades HTTP to HTTPS if the website supports it unlike Chrome, but it's not like you can't visit HTTP websites on Firefox. Blocking HTTP sites is not the default, and it shouldn't be IMO. There must be something else going on.
All/many of the recent Anti-Phishing tests of AVC has been commisioned by Avast. They know that their product is very effective against phishing so that's why they do it I guess. But since now they are owned by Norton aka Gen Digital, now it says that "Commissioned by Gen Digital". When you hear this new Gen Digital name, you would first think of Norton before any other brand under their umbrella, and this test makes Norton looks so bad. It did even worse than Firefox and Edge.
May have something to do with rating new, low reputation sites in yellow and not blocking them completely.
I tried to make them change so many of the sites that they rated Suspicious instead of blocking them, but they didn't change the rating for most sites that I submitted. But I also faced more false positives with Norton. But it's at least better than submitting to SmartScreen. SmartScreen never blacklisted anything I submitted over the years, Google with Safe Browsing did many times.
Looks like according to Firefox Google Safe Browsing is their only source. Don't see anything else here:
 

Trident

Level 27
Verified
Top Poster
Well-known
Feb 7, 2023
1,629
I always prefer the built-in browsers and software. My philosophy before installing a program has always been
-Do you really need it?
-Do you have software that offers similar capabilities already?
-Is there notable benefit over what’s already there?

In the light of this, I use Safari on Mac and iOS(phishing protection in Safari is a disgrace whilst we’re at it), Chrome on Chrome OS and Edge on Windows. Firefox doesn’t fit anywhere for me.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top