AV-Comparatives Anti-Phishing Protection in AVs and Browsers – February 2023 - commissioned by Gen Digital

[Gandalf_The_Grey]

Introduction
This test evaluates the phishing-page detection rates and false positive rates at time of testing (February 2023) of various AV products, and different browsers, as listed below. AV-Comparatives selected the products to be tested. We purchased these in January 2023. This comparative test was commissioned by Gen Digital. The testing dates were chosen by the testing lab and not pre-disclosed.

Tested products
The test took place from 1st February till 19th February 2023. The following up-to-date products were tested in parallel, and with active Internet/cloud-access, on Windows 10. Phishing protection provided by AV products were tested using Google Chrome (with Google SafeBrowsing disabled). The browser extensions of the AV products were installed and enabled. The different browsers were tested without an AV program running.

Antivirus (AV) Products:
• Avast Free Antivirus 22.12 – 23.1
• Avira Free Security 1.1
• Bitdefender Internet Security 26.0
• ESET Internet Security 16.0
• Kaspersky Standard 21.8
• Malwarebytes Premium 4.5
• McAfee Total Protection 16.0
• Microsoft Defender 4.18 with Defender browser plugin for Chrome
• Norton 360 Deluxe 22.23
• Trend Micro Internet Security 17.7

Browsers:
• Avast Secure Browser 109.0 – 110.0
• Google Chrome 109.0 – 110.0 with SafeBrowsing
• Microsoft Edge 109.0 – 110.0
• Mozilla Firefox 109.0 – 110.0
• Opera 95.0

Anti-Phishing Test-Set
For this first test, we used 250 valid phishing URLs. The number of used clean URLs for false alarm detection was 250.

Test Results

AV-Comparatives: Anti-Phishing Protection in AVs and Browsers – February 2023 - AMTSO

www.amtso.org

PDF:

https://www.av-comparatives.org/wp-content/uploads/2023/03/gen_phishing_02-2023.pdf

 

[nickstar1]

Look at that Mcafee score!!! they have much improved but waiting for the new and improved version which is supposed to cut all the junk out of the product so it will be even better. To get a better score then Kaspersky is impressive.

 

[Trident]

Look at that Mcafee score!!! they have much improved

McAfee Web Advisor was always above average, now has it been better than Kaspersky is hard to tell. But it was one of the first technology, if not the first one to block and rate malicious websites.

What’s going on at Norton, what is this poor score? Compromised 0.4% on Real World protection, second lowest score on the phishing test… May have something to do with rating new, low reputation sites in yellow and not blocking them completely.

 

[ForgottenSeer 97327]

@Trident Indeed,

Norton does not make any sense. Norton should get URL and sample feeds from Avast, Avira, AVG, BullGuard and Symantec. In theory it should have the largest / broadest malware intelligence network.

???

 

[Trident]

@Trident Indeed,

Norton does not make any sense. Norton should get URL and sample feeds from Avast, Avira and AVG and business solution Symantec Endpoint Protection. In theory it should have the largest / broadest malware intelligence network.

???

I don’t know what they are doing, but since detection varies (like on Real World Protection Norton and Avira had 0.4% misses and Avast had none), as well as looking at this score where Avast emerges as second best and Norton as second worst, it looks like Norton has not integrated any threat feeds whatsoever. I don’t know why this is and for what they sprinkled 360 million + 8 billion.

 

[ForgottenSeer 97327]

Another result surprises me: can someone explain why Firefox scored better than Google when it uses the 'slow 30 minute update but private' version of Google safe browsing?

 

[Trident]

Another result surprises me: can someone explain why Firefox scored better than Google when it uses the '30 minute update but private' version of Google safe browsing?
View attachment 274876

Though it uses Google Safe Browsing, there is no guarantee it doesn’t use any heuristics to detect brand impersonation or any URL-reputation logics. This is where the difference may be coming from.

 

[ForgottenSeer 97327]

Though it uses Google Safe Browsing, there is no guarantee it doesn’t use any heuristics to detect brand impersonation or any URL-reputation logics. This is where the difference may be coming from.

Other browsers also have typo protection. Considering the reach and the experience both Microsoft and Google have in AI,webcrawling and the combination of having a browser plus a search engine, it is very unlikely Mozilla would have been able out smarten them.

The only reason I can think of is that Firefox had blocking of HTTP websites enabled (80% of phishing and scam URL's are unecrypted).

 

[Trident]

Other browsers also have typo protection. Considering the reach and the experience both Microsoft and Google have in AI,webcrawling and the combination of having a browser plus a search engine, it is very unlikely Mozilla would have out smartened them.

The only reason I can think of is that Firefox had blocking of HTTP websites enabled (80% of phishing and scam URL's are unecrypted).

That may be one reason, also (I’ve not installed Chrome in years so I may be wrong) but by default it gets installed with the enhanced security turned off, doesn’t it? As far as I remember, this relies only on a local copy of the safe browsing cache and doesn’t check every URL in real time.

Manage Enhanced Safe Browsing for your account - Google Account Help

Enhanced Safe Browsing for your account can help provide more protection against phishing and malware across Google Chrome and Gmail. Manage Enhanced Safe Browsing

support.google.com

I am inclined to believe this score as on all my tests Chrome has always performed very poorly against phishing and malicious sites, but haven’t ever tested Firefox so not sure if it’s better. For me this browser is non-existent.

 

[SeriousHoax]

The only reason I can think of is that Firefox had blocking of HTTP websites enabled (80% of phishing and scam URL's are unecrypted).

I'm surprised by the difference between Chrome and Firefox also but this is not the first time. I have seen massive difference like this in some other professional tests also. Firefox by default auto upgrades HTTP to HTTPS if the website supports it unlike Chrome, but it's not like you can't visit HTTP websites on Firefox. Blocking HTTP sites is not the default, and it shouldn't be IMO. There must be something else going on.
All/many of the recent Anti-Phishing tests of AVC has been commisioned by Avast. They know that their product is very effective against phishing so that's why they do it I guess. But since now they are owned by Norton aka Gen Digital, now it says that "Commissioned by Gen Digital". When you hear this new Gen Digital name, you would first think of Norton before any other brand under their umbrella, and this test makes Norton looks so bad. It did even worse than Firefox and Edge.

May have something to do with rating new, low reputation sites in yellow and not blocking them completely.

I tried to make them change so many of the sites that they rated Suspicious instead of blocking them, but they didn't change the rating for most sites that I submitted. But I also faced more false positives with Norton. But it's at least better than submitting to SmartScreen. SmartScreen never blacklisted anything I submitted over the years, Google with Safe Browsing did many times.
Looks like according to Firefox Google Safe Browsing is their only source. Don't see anything else here:

How does built-in Phishing and Malware Protection work? | Firefox Help

Firefox contains built-in Phishing and Malware Protection to help keep you safe online. This article explains how they work.

support.mozilla.org

 

[Jan Willy]

For me this browser is non-existent.

Why?

 

[Trident]

Why?

I always prefer the built-in browsers and software. My philosophy before installing a program has always been
-Do you really need it?
-Do you have software that offers similar capabilities already?
-Is there notable benefit over what’s already there?

In the light of this, I use Safari on Mac and iOS(phishing protection in Safari is a disgrace whilst we’re at it), Chrome on Chrome OS and Edge on Windows. Firefox doesn’t fit anywhere for me.