Hot Take [Updated 29/12/2018] Browser extension comparison: Malwares and Phishings

[Evjl's Rain]

Comparison between browser extensions

Test 29/12
Q&A - [Updated 29/12/2018] Browser extension comparison: Malwares and Phishings

Test 24/11
Q&A - [Updated 24/11/2018] Browser extension comparison: Malwares and Phishings

Test 12/11
Q&A - [Updated 12/11/2018] Browser extension comparison: Malwares and Phishings

Test 7/11
Q&A - [Updated 7/11/2018] Browser extension comparison: Malwares and Phishings

Test 6/9
Q&A - [Updated 3/9/2018] Browser extension comparison: Malwares and Phishings

Test 3/9
Q&A - [Updated 3/9/2018] Browser extension comparison: Malwares and Phishings

Test 2/9
Q&A - [Updated 25/7/2018] Browser extension comparison: Malwares and Phishings

Test, quick 1/9
Q&A - [Updated 25/7/2018] Browser extension comparison: Malwares and Phishings

Fun test 25/7/2018
Q&A - [Updated 24/7/2018] Browser extension comparison: Malwares and Phishings

Updated 24/7/2018 (most comprehensive, as possible)
Q&A - [Updated 24/7/2018] Browser extension comparison: Malwares and Phishings

Updated 19/7/2018
Q&A - [Updated 10/7/2018] Browser extension comparison: Malwares and Phishings

Updated 18/7/2018
Q&A - [Updated 10/7/2018] Browser extension comparison: Malwares and Phishings

Updated 10/7/2018
Q&A - [Updated 10/7/2018] Browser extension comparison: Malwares and Phishings

Updated 7/6/2018
Q&A - [Updated 7/6/2018] Browser extension comparison: Malwares and Phishings

Updated 3/6/2018
Q&A - [Updated 3/6/18] Browser extension comparison: Malwares and Phishings

Updated 25/4/2018
Poll - [Updated 25/4/18] Browser extension comparison: Malwares and Phishings

Update: 23/3/2018
Poll - [Updated 23/3/18] Browser extension comparison: Malwares and Phishings


Browser: Google Chrome 65 x64
Malware and phishing links: 10 malc0de, 10 vxvault, 10 openphish, 10 verified phishtank, 10 unverified phishtank
Total: 50 links

Spoiler: Links

Dropbox - link test 17-3-2018.txt

Extensions: recently downloaded from Chrome Web Store
- Google Safe Browsing (built-in chrome's protection)
- AdGuard AdBlocker: default settings, uses Google Safe Browsing (delayed) and their own database
- Avira browser safety: default settings
- Norton Safe Web: default settings
- Bitdefender Trafficlight: default settings, it rarely blocks any malware links, just old ones
- Avast Online Security: default settings, only has phishing protection, expected to score 0 against malwares
- Netcraft Extension: default settings, only has phishing protection, expected to score 0 against malwares
- uBlock Origin with some additional filters

Spoiler: ublock origin filters

http://hosts-file.net/emd.txt
http://hosts-file.net/exp.txt
http://hosts-file.net/pup.txt
http://hosts-file.net/psh.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/RW_URLBL.txt

NOTE: the result can vary from day-to-day. Tomorrow with different links, the result can be very different. All are live links but they can be dead a few minutes after the test. No duplication

Results:

Winner: Google Safe Browsing

 

[Evjl's Rain]

@Evjl's Rain norton extension is the new one you used? If yes, it seems they just made it worse

there is only one and it is. Norton is weak nowadays
the new one just adds isolation for blocked sites but not for missed sites and the number of blocked sites are low => not useful
not sure how to test it properly but it has been declining significantly compared to last year

 

[imuade]

Test 6/9/2018, all from VT 60 links

Forticlient 53/60

Great job, as usual
Which settings did you use for Forticlient? Default (Malicious, Phishing, Spam URLs, Dynamic DNS) ?
I have set FC to warn about "Newly Observed Domain" and "Newly Registered Domain" too, this is especially good for phishing websites

 

[Evjl's Rain]

Great job, as usual
Which settings did you use for Forticlient? Default (Malicious, Phishing, Spam URLs, Dynamic DNS) ?
I have set FC to warn about "Newly Observed Domain" and "Newly Registered Domain" too, this is especially good for phishing websites

indeed it will be great but it increases FP rate too
I prefer using default settings for all products if possible for comparable results unless the default settings are really bad

 

[imuade]

indeed it will be great but it increases FP rate too
I prefer using default settings for all products if possible for comparable results unless the default settings are really bad

Yeah, I agree, that's why I set FC to warn instead of block (with "warn" you still get the block page, but you have the option to proceed anyway)

 

[Evjl's Rain]

Tested out Adguard for desktop/windows malware protection, which they claim to use Google Safe Browsing API v2.2 (ancient) and their own Lookup API
vxvault 101 links, ~ half of them are live, the rest are dead
Google chrome (true google safe browsing v4): blocked all live malwares/links
Adguard: missed 37 malwares, blocked 15

 

[Gandalf_The_Grey]

Tested out Adguard for desktop/windows malware protection, which they claim to use Google Safe Browsing API v2.2 (ancient) and their own Lookup API
vxvault 101 links, ~ half of them are live, the rest are dead
Google chrome (true google safe browsing v4): blocked all live malwares/links
Adguard: missed 37 malwares, blocked 15

Love AdGuard (extension) as adblocker, but don't use it for malware protection.
Thanks for your testing

 

[Windows_Security]

URL blocking is a numbers game.

Google and Microsoft should do well because they simply see the most traffic. Corporate clients oriented UTM suppliers (Fortinet, Sophos, Symantec) and DNS networks (Neustar/Cisco/Comodo/Quad9) also 'see' a lot of traffic as with largest AV-vendors (Bitdefender, AVAST/AVG and Kaspersky).

In the past we could smartly stack/pile some vendors: a (free) DNS service, your AntiVirus and browser of choice, topped with an extension. Unfortunately the tests of @Evjl's Rain show that using different layers in different phases of the network stack, is not worth the effort anymore.

Based on these disappointing test results I returned to using my browser's URL filter only, no blocking extensions and back to ISP's DNS service. But I will keep monitoring this thread, just in case stacking up is beneficial again

 

[Moonhorse]

URL blocking is a numbers game.

Google and Microsoft should do well because they simply see the most traffic. Corporate clients oriented UTM suppliers (Fortinet, Sophos, Symantec) and DNS networks (Neustar/Cisco/Comodo/Quad9) also 'see' a lot of traffic as with largest AV-vendors (Bitdefender, AVAST/AVG and Kaspersky).

In the past we could smartly stack/pile some vendors: a (free) DNS service, your AntiVirus and browser of choice, topped with an extension. Unfortunately the tests of @Evjl's Rain show that using different layers in different phases of the network stack, is not worth the effort anymore.

Based on these disappointing test results I returned to using my browser's URL filter only, no blocking extensions and back to ISP's DNS service. But I will keep monitoring this thread, just in case stacking up is beneficial again

Hard to say, its up to user. I can bloat my chrome with many extensions without having performance issues and i kinda find these extensions as must have:
- netcraft ( phishing protection better than most avs)
- Malwarebytes ( against pups, malware, scams) disable ad / tracking protection
- antivirus do web filtering again everything
- Neustar dns is just extra layer of defense + faster browsing than isp on most cases

But like having avast web protection + malwarebytes extension is example of useless stacking

After all youre fine with adblocker + medium mode of ublock + password manager as example is more than enough these days. You never will type password manually in this case, and blocking all 3rd party scripts throught ublock you wont run into any problems, this is just common sense but still there are people not advanced to do this kind of easy stuff

On firefox i dont like to stack all these extensions because of ram usage, and just go with adblocker and let firefox do rest

I think open social media accounts, that aint private is bigger problem this day. Past weekend i have read alot of cases of so called doxing, like gathering personal info throught social media accounts and using that info to recover/brute force other accounts, emails as example...wich can lead into huge problems.

 

[Andy Ful]

Tested out Adguard for desktop/windows malware protection, which they claim to use Google Safe Browsing API v2.2 (ancient) and their own Lookup API
vxvault 101 links, ~ half of them are live, the rest are dead
Google chrome (true google safe browsing v4): blocked all live malwares/links
Adguard: missed 37 malwares, blocked 15

Could you test Exploit Guard 'Network Protection' which is built-in on Windows 10 ver. 1709+?
I made a simple test to prove that 'Network Protection' really works, and could be compared to Edge SmartScreen anti-phishing filter. But, I used only 20 live samples and did not test the reverse relationship.
Q&A - ConfigureDefender utility for Windows 10
As Microsoft claims, the 'Network Protection' is the system-wide feature related to SmartScreen:

"Network protection helps reduce the attack surface of your devices from Internet-based events. It prevents employees from using any application to access dangerous domains that may host phishing scams, exploits, and other malicious content on the Internet.
It expands the scope of Windows Defender SmartScreen to block all outbound HTTP(s) traffic that attempts to connect to low-reputation sources (based on the domain or hostname)."

Thanks.

Edit.
Personally, I use Adguard DNS + Exploit Guard 'Network Protection' without any issues.

 

[Evjl's Rain]

Could you test Exploit Guard 'Network Protection' which is built-in on Windows 10 ver. 1709+?
I made a simple test to prove that 'Network Protection' really works, and could be compared to Edge SmartScreen anti-phishing filter. But, I used only 20 live samples and did not test the reverse relationship.
Q&A - ConfigureDefender utility for Windows 10
As Microsoft claims, the 'Network Protection' is the system-wide feature related to SmartScreen:

"Network protection helps reduce the attack surface of your devices from Internet-based events. It prevents employees from using any application to access dangerous domains that may host phishing scams, exploits, and other malicious content on the Internet.
It expands the scope of Windows Defender SmartScreen to block all outbound HTTP(s) traffic that attempts to connect to low-reputation sources (based on the domain or hostname)."

Thanks.

Edit.
Personally, I use Adguard DNS + Exploit Guard 'Network Protection' without any issues.

it was a very difficult test to do
after 30mins, I finally got the result. Tested several times with similar results.
vxvault: 101 links, 33 live links
WD's network protection blocked 15 links
WDBP extension blocked 55 links, including dead ones

Realtime protection on/off or folder exclusion didn't affect the result

the problem was WD's network protection didn't show notification for every blocked link, they were grouped
I decided to count the number of missed downloads
network protection missed 18, total downloads with the protection were 33

 

[Andy Ful]

I noticed on my test, that 'Network Protection' always blocked malicious files from websites, for example:
olkamo-xxxxxxx.com/favicon.ico
spaparne.xx/xxxxxx/slideshow/slide1.jpg
So, it may be not easy to compare it with web browser extensions, which usually blocks the websites:
olkamo-xxxxxxx.com
spaparne.gr

the problem was WD's network protection didn't show notification for every blocked link, they were grouped
I decided to count the number of missed downloads
network protection missed 18, total downloads with the protection were 33

In some cases the malicious files can be removed from the website and then 'Network Protection' will not block anything. On the contrary, the web browser extensions may remove the website from the blacklist after several days (weeks).
Is that why you counted total missed downloads?

 

[Evjl's Rain]

In some cases the malicious files can be removed from the website and then 'Network Protection' will not block anything. On the contrary, the web browser extensions may remove the website from the blacklist after several days (weeks).
Is that why you counted total missed downloads?

I counted missed files because they are working links. WD should block them
if the links don't work, nothing will be downloaded and thus no block message
I noticed WD network protection only blocked working links and didn't block dead links
however, 15/33 is not a high number

When I turned on the realtime protection, WD detected most of the downloaded files so they are malicious

I think WDNP is not the same as smartscreen and I don't know how it works correctly
if it does use smartscreen, why didn't it block the same number of files that WD browser protection blocked (lower number of missed files)? What kind of smartscreen it it? Why is Windows smartscreen different from SS of WDBP and WDNP
I do believe it's in its early stage of development so it can't be compared to smartscreen or WDBP

 

[Andy Ful]

I counted missed files because they are working links. WD should block them
if the links don't work, nothing will be downloaded and thus no block message
I noticed WD network protection only blocked working links and didn't block dead links
however, 15 is not a high number

When I turned on the realtime protection, WD detected most of the downloaded files so they are malicious

I think WDNP is not the same as smartscreen and I don't know how it works correctly
if it does use smartscreen, why didn't it block the same number of files that WD browser protection blocked (lower number of missed files)? What kind of smartscreen it it? Why is Windows smartscreen different from SS of WDBP and WDNP
I do belive it's in its early stage of development so it can't be compared to smartscreen or WDBP

Definitely 'Network Protection' on Windows 10 is not the same as SmartScreen on IE or Edge or Windows Defender Browser Protection .
The first, can block mostly malicious files on websites the others blocks websites. So using the first, the user can browse the legal website which has been hacked - only the malicious content (*.ico, *.jpg, etc.) is blocked. This can be probably a disadvantage when we have a truly malicious website which malicious content is changing frequently. In fact, both 'Network Protection' and SmartScreen anti-phishing filter in Edge (IE) can use SmartScreen Reputation Cloud, but in a different way. For example, every website which had something blocked by 'Network Protection' will also be blocked by SmartScreen in Edge for some time, even when the malicious file is removed from the website.
If so, then the detection of SmartScreen in Edge will be always better for truly malicious websites and 'Network Protection' will always produce less false positives on legal hacked websites.

 

[Brie]

Test 6/9/2018, all from VT 60 links


chrome 49/60
ublock (custom) 35/60 => all blocks were from hphosts (full + partial = EMD+PSH+HJK+...)
avira 2/60
blocksi (block unrated) 60/60
comodo 0/60
malwarebytes 39/60
WDBP 4/60
Norton 3/60
Edge/IE 19/59

Forticlient 53/60
Kaspersky 53/60
K9 (default/custom = same result) 59/60

Thor RC 18/60
hphosts 35/60
Missed item analysis:
- 19 items are identical
- 5 items are exclusive in hphosts, 24 are in Thor RC
View attachment 197107

ublock custom filters
View attachment 197106

k9 custom filters
View attachment 197108

these keep changing in effectiveness. i just uninstalled k9. now it tests good. ????????

 

[Andy Ful]

Evjl's Rain,
Thanks for testing. I very much appreciate the effort you put in your tests.:emoji_ok_hand:

 

[Brie]

hi

i also value your testing evjl's rain. thank you very much for your efforts.

 

[Windows_Security]

May be I am remembering it wrong, but I think to have seen network protection also blocking links from mail. I remember to have seen it both on my wife's 2-1 laptop (outlook 2016) and on my Asus Transformer (mail + pre-installed home office 2013).

 

[Evjl's Rain]

these keep changing in effectiveness. i just uninstalled k9. now it tests good. ????????

sorry I don't understand what you mean
k9 is still the best in this test
blocksi is just a default-deny blocker which has so many problems that it can block safe websites if they are not in blocksi's database
it also slows down browsing speed
moreover, K9 also has the same option which can block unrated websites thus blocks everything

 

[Evjl's Rain]

@Andy Ful I just tested it with 100 openphish links (not as new as phishtank)
it blocked only 1 link

 

[HarborFront]

Test 6/9/2018, all from VT 60 links

chrome 49/60
ublock (custom) 35/60 => all blocks were from hphosts (full + partial = EMD+PSH+HJK+...)
avira 2/60
blocksi (block unrated) 60/60
comodo 0/60
malwarebytes 39/60
WDBP 4/60
Norton 3/60
Edge/IE 19/59

Forticlient 53/60
Kaspersky 53/60
K9 (default/custom = same result) 59/60

Thor RC 18/60
hphosts 35/60
Missed item analysis:
- 19 items are identical
- 5 items are exclusive in hphosts, 24 are in Thor RC
View attachment 197107

ublock custom filters
View attachment 197106

k9 custom filters
View attachment 197108

So what are the strengths of K9 over uBO and Malwarebytes and vice versa?

Thanks