Hot Take [Updated 29/12/2018] Browser extension comparison: Malwares and Phishings

[Evjl's Rain]

Comparison between browser extensions

Test 29/12
Q&A - [Updated 29/12/2018] Browser extension comparison: Malwares and Phishings

Test 24/11
Q&A - [Updated 24/11/2018] Browser extension comparison: Malwares and Phishings

Test 12/11
Q&A - [Updated 12/11/2018] Browser extension comparison: Malwares and Phishings

Test 7/11
Q&A - [Updated 7/11/2018] Browser extension comparison: Malwares and Phishings

Test 6/9
Q&A - [Updated 3/9/2018] Browser extension comparison: Malwares and Phishings

Test 3/9
Q&A - [Updated 3/9/2018] Browser extension comparison: Malwares and Phishings

Test 2/9
Q&A - [Updated 25/7/2018] Browser extension comparison: Malwares and Phishings

Test, quick 1/9
Q&A - [Updated 25/7/2018] Browser extension comparison: Malwares and Phishings

Fun test 25/7/2018
Q&A - [Updated 24/7/2018] Browser extension comparison: Malwares and Phishings

Updated 24/7/2018 (most comprehensive, as possible)
Q&A - [Updated 24/7/2018] Browser extension comparison: Malwares and Phishings

Updated 19/7/2018
Q&A - [Updated 10/7/2018] Browser extension comparison: Malwares and Phishings

Updated 18/7/2018
Q&A - [Updated 10/7/2018] Browser extension comparison: Malwares and Phishings

Updated 10/7/2018
Q&A - [Updated 10/7/2018] Browser extension comparison: Malwares and Phishings

Updated 7/6/2018
Q&A - [Updated 7/6/2018] Browser extension comparison: Malwares and Phishings

Updated 3/6/2018
Q&A - [Updated 3/6/18] Browser extension comparison: Malwares and Phishings

Updated 25/4/2018
Poll - [Updated 25/4/18] Browser extension comparison: Malwares and Phishings

Update: 23/3/2018
Poll - [Updated 23/3/18] Browser extension comparison: Malwares and Phishings


Browser: Google Chrome 65 x64
Malware and phishing links: 10 malc0de, 10 vxvault, 10 openphish, 10 verified phishtank, 10 unverified phishtank
Total: 50 links

Spoiler: Links

Dropbox - link test 17-3-2018.txt

Extensions: recently downloaded from Chrome Web Store
- Google Safe Browsing (built-in chrome's protection)
- AdGuard AdBlocker: default settings, uses Google Safe Browsing (delayed) and their own database
- Avira browser safety: default settings
- Norton Safe Web: default settings
- Bitdefender Trafficlight: default settings, it rarely blocks any malware links, just old ones
- Avast Online Security: default settings, only has phishing protection, expected to score 0 against malwares
- Netcraft Extension: default settings, only has phishing protection, expected to score 0 against malwares
- uBlock Origin with some additional filters

Spoiler: ublock origin filters

http://hosts-file.net/emd.txt
http://hosts-file.net/exp.txt
http://hosts-file.net/pup.txt
http://hosts-file.net/psh.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt
https://ransomwaretracker.abuse.ch/downloads/RW_URLBL.txt

NOTE: the result can vary from day-to-day. Tomorrow with different links, the result can be very different. All are live links but they can be dead a few minutes after the test. No duplication

Results:

Winner: Google Safe Browsing

 

[Evjl's Rain]

too much filters can slow down your loading/working web browser.

true
I always keep my total number of filters <100k
I used to have 400k but I didn't really see any slowdown but memory leak in some cases
my current setup works best for me, I don't use english or easylist

 

[Evjl's Rain]

@Evjl's Rain ... most appreciated! Thanks a lot.

As I said, I don't use UMatrix/UBlock anymore.
However, your interesting test comparisons here in this post, and your UBlock's defense as your personal best choice... positively motivated me to run more tests on UMatrix/UBlock. Perhaps your hosts file combo is the "silver bullet".

I will run my tests on this or maximum on the next week, comparing with Malwarebytes add-on. I also will compare with the brand new Comodo add-on (it appeared today). And also Avira, Bitdefender etc will be included on my tests.

Thank you again for the hosts info!

PS: I received answer from Malwarebytes. They said they use cloud scanner + heuristics + own data base... they don't use 3rd-party hosts files.

malwarebytes doesn't use hosts file but cloud, of course
but they have the same database and they block the same thing. MB extension just blocks a bit more. I think MB extension and MB for windows use the same database on cloud. Same as other extensions like avira, BD, avast and norton,...
hphosts is source where we can access malwarebytes's database and implement it to our filters
cloud querying is faster and require less resource

I tested the comodo extension. It's an absolute garbage. It blocked 0 in my test, absolute 0

your approach of blocking third-party is great
I tried it once and it worked perfectly
however, I visit a lot of websites and they require too many manual whitelisting unless they would break
I gave up. Back to the traditional approach of blacklisting

if it works for you, you don't really need a lot filters because blocking third-party is the best filter

 

[Decopi]

Hi @Evjl's Rain !

I did now a very quick informal test, using your hosts files combo for UMatrix/UBlock.
I compared UBlock, Bitdefender, Avira, Malwarebytes, Safebrowsing google, Netcraft and Norton. Each one in a different separate browser (Firefox).

For phishing... 100% agree with you that Netcraft and Safebrowsing google are the best.
Norton was the worst.

For malware... I got results very different from yours.
Norton was a complete disaster. The worst. In your tests was the best.
Bitdefender also was very bad.
Malwarebytes was the best in my tests for malware.

My conclusion is not that you are right or wrong.
I believe there is a problem with our tests.
I believe that different samples and different URLs, always will show different test results.
Even same source of samples, will give different test results for same add-on, for example on different sample dates. Also as you mentioned in your comment, even same source of samples, and same URL or sample, might give different test results because links change every hour.
There is a problem with the test methodology (at least, with my methodology). I need to think in another different way to test these add-ons. I will try to do that next week.

Cheers

 

[Evjl's Rain]

Hi @Evjl's Rain !

I did now a very quick informal test, using your hosts files combo for UMatrix/UBlock.
I compared UBlock, Bitdefender, Avira, Malwarebytes, Safebrowsing google, Netcraft and Norton. Each one in a different separate browser (Firefox).

For phishing... 100% agree with you that Netcraft and Safebrowsing google are the best.
Norton was the worst.

For malware... I got results very different from yours.
Norton was a complete disaster. The worst. In your tests was the best.
Bitdefender also was very bad.
Malwarebytes was the best in my tests for malware.

My conclusion is not that you are right or wrong.
I believe there is a problem with our tests.
I believe that different samples and different URLs, always will show different test results.
Even same source of samples, will give different test results for same add-on, for example on different sample dates. Also as you mentioned in your comment, even same source of samples, and same URL or sample, might give different test results because links change every hour.
There is a problem with the test methodology (at least, with my methodology). I need to think in another different way to test these add-ons. I will try to do that next week.

Cheers

hi, thank you for your test
could you please provide us the malware list you used in your test? are they .exe files or just domains and IPs?
I use malc0de and vxvault, which are extremely popular and many vendors have got the signatures for them already so this might affect the real result. In the other hand, if a vendor does very badly in my malware link test, it means it's bad because malwares from vxvault and malc0de are widespread or many people have been infected by them

 

[Decopi]

@Evjl's Rain ... I tried to emulate your test (as much as possible). I tried to use your hosts for UBlock, and your phishing/malware sources (I took from your first comment in this post, describing your test). I used 10 samples for each source. I mixed the samples, taking a few from newest on the list, other from the middle, and others from the end of the list (on same day).

But again @Evjl's Rain , this was a very sloppy quick informal test from my side. I need to run something more formal, perhaps on next week.

At first sight, I have some unproven conclusions:
1) Test methodology need to be improved (at least from my side).
2) It does not exist the perfect solution, or the perfect add-on etc. A combination of tools seems to be better.
3) Tools/Add-ons must be tested considering their strengths. For example, Netcraft x Safebrowsing for phishing, Norton x Bitdefender for malwares etc. I really don't see much sense in comparing everything with everything.
4) Also, tools/add-ons must be tested considering "zero-day" samples x 1 day old samples... because test results are dramatically different even in same source
5) Browsers must be updated, and also add-ons must be with data-bases updated. This is not so simple to be done. An add-on data-base not updated, will show very different test results.
6) Etc

 

[Evjl's Rain]

A quick test with 20 links from malc0de and vxvault

Spoiler: link

Dropbox - link 25-4-18.txt

I attached the links to the filters used. Click on these links if you want to view

Individual tests:
GSB: 13/20
Norton: 12/20
Bitdefender Trafficlight: 1/20
Avira: 7/20
ublock default: 2/20
ublock+ADZ: 6/20
ublock+squidblack: 11/20

ublock + hphosts full + hphosts partial: 17/20
Malwarebytes: 16/20
MB scored 16 while hphosts scored 17 => missed link was classified in ad/tracker category (ATS) despite having a malware

WDBP: 7/20
Edge: 17/20

WDBP 2nd test: 7/20
WDBP 3rd test: 4/20
WDBP 4th test: 7/20
WDBP 5th test: 4/20
=> Windows defender browser protection is inconsistent. Still in Beta

Microsoft Edge also warns users about files without verified signatures => I chose the recommended option to block the files from being downloaded. Therefore, better result

Combined tests:
GBS+norton+WDBP+MB: 18/20
GBS+norton+WDBP+MB+uBO squidblack: 18/20

My setup: GBS+norton+WDBP+MB+uBO squidblack+1hosts (+Stevenblack = blocked 0/20): 19/20

Winner: Malwarebytes and Microsoft Edge

 

[shmu26]

A quick test with 20 links from malc0de and vxvault

Spoiler: link

Dropbox - link 25-4-18.txt

I attached the links to the filters used. Click on these links if you want to view

Individual tests:
GSB: 13/20
Norton: 12/20
Bitdefender Trafficlight: 1/20
Avira: 7/20
ublock default: 2/20
ublock+ADZ: 6/20
ublock+squidblack: 11/20

ublock + hphosts full + hphosts partial: 17/20
Malwarebytes: 16/20
MB scored 16 while hphosts scored 17 => missed link was classified in ad/tracker category (ATS) despite having a malware

WDBP: 7/20
Edge: 17/20

WDBP 2nd test: 7/20
WDBP 3rd test: 4/20
WDBP 4th test: 7/20
WDBP 5th test: 4/20
=> Windows defender browser protection is inconsistent. Still in Beta

Microsoft Edge also warns users about files without verified signatures => I chose the recommended option to block the files from being downloaded. Therefore, better result

Combined tests:
GBS+norton+WDBP+MB: 18/20
GBS+norton+WDBP+MB+uBO squidblack: 18/20

My setup: GBS+norton+WDBP+MB+uBO squidblack+1hosts (+Stevenblack = blocked 0/20): 19/20

Winner: Malwarebytes and Microsoft Edge

Pardon my ignorance, but how to use hphosts full + hphosts partial ?
Is this a list you add to the hosts file, or what?
Then you need to update it manually every once in a while?

 

[Evjl's Rain]

Pardon my ignorance, but how to use hphosts full + hphosts partial ?
Is this a list you add to the hosts file, or what?
Then you need to update it manually every once in a while?

no I added them to ublock origin only. ublock supports hosts format and auto-updates every 7 hours by default
Adding these 2 (~600k domains) to the hosts is a disaster

 

[Stas]

Evjl's Rain did you disable advertising/tracker protection in Malwarebytes protection settings?

 

[Evjl's Rain]

Evjl's Rain did you disable advertising/tracker protection in Malwarebytes protection settings?

I tested both. First, left everything default, then disabled the tracking protection
it didn't affect the result
perhaps, because ad/tracking protection doesn't show the block message like the other options
it just shows the number of blocked ads on the extension icon

 

[Stas]

I tested both. First, left everything default, then disabled the tracking protection
it didn't affect the result
perhaps, because ad/tracking protection doesn't show the block message like the other options
it just shows the number of blocked ads on the extension icon

I mean if you use uBO+Malwarebytes extension you don't need advertising/tracker protection in Malwarebytes. Did you notice any change in resources when disabling advertising/tracker protection in Malwarebytes? Is it lighter?

 

[Evjl's Rain]

I mean if you use uBO+Malwarebytes extension you don't need advertising/tracker protection in Malwarebytes. Did you notice any change in resources when disabling advertising/tracker protection in Malwarebytes? Is it lighter?

I always disable that option because it would definitely conflict with ublock somehow

I quickly tested checking and unchecking that option with 25 links opened simultaneously. I monitored using Chrome's builtin task manager
CPU usage
unchecked: 30%, peak 40%
checked: 30%, peak 67%
so it certainly affect something but not really noticeable

 

[tonibalas]

@Evjl's Rain is Malwarebytes extension a good addition on a browser?
Have tested it alongside Adguard extension to see if there are any conflicts?

 

[Evjl's Rain]

@Evjl's Rain is Malwarebytes extension a good addition on a browser?
Have tested it alongside Adguard extension to see if there are any conflicts?

it's now a must-have for me
it's very powerful and has heuristics which are not seen in other extensions
I saw it didn't block a website but after 2-3 seconds, MB blocked it. Normally, it blocked before I could even see the page

test: the same website (not found in database), hphosts can't block while MB blocks => heuristics do exist

it does conflict with adguard/ublock so you should uncheck "Enable advertising/tracker protection" in settings
it did conflict with ublock only once because they are trying to block the same link (I was testing against malwares) but it doesn't matter, just ignore it

it conflicted with Smart HTTPS

This extension failed to redirect a network request to chrome-extension://ihcjicgdanjaechkgeegckofjjedodee/redirects/blockedMalware.html?url=http%3A%2F%2Fwww.vn-zoom.com%2F&amp;host=www.vn-zoom.com&amp;type=malware&amp;subtype=malware because another extension (Smart HTTPS) redirected it to https://www.vn-zoom.com/.
RELOAD

 

[tonibalas]

@Evjl's Rain thank so much for your quick answer.
I will try MB extension.
I just hope it wont slow down my browsing too much because my laptop is old.
When i see how it works i will let you know.

 

[shmu26]

HTTPS Everywhere
This extension failed to redirect a network request to https://www.google-analytics.com/analytics.js because another extension (uBlock Origin) redirected it to ch

I have been seeing this a lot lately, and I don't have malwarebytes extension, only ublock extension.

 

[Arequire]

HTTPS Everywhere
This extension failed to redirect a network request to https://www.google-analytics.com/analytics.js because another extension (uBlock Origin) redirected it to ch

I have been seeing this a lot lately, and I don't have malwarebytes extension, only ublock extension.

It's a known conflict. Gorhill says it doesn't actually affect anything. As far as I'm aware neither Gorhill or EFF can do anything about it; it's on the Chromium devs to provide a solution.

 

[Decopi]

Hi @Evjl's Rain !

I am glad you liked Malwarebytes add-on.

I am using both, Malwarebytes + Netcraft... unbeatable! Both catches what the others missed (including antivirus/antimalware failures).

My system performance was improved because I am not using UMatrix, UBlock, hosts, I am not even using the browser built-in anti-tracking. Zero! Nothing.

For privacy/anti-tracking I have a tiny lightweight 3rd-party blocker.
Also, I use FPI (First Party isolation) and Containers.
So, tracking in my browser is reduced 90%.

Till today is the best combo I found, improving performance, security, privacy, without system impact (RAM, CPU, internet speed).

 

[tonibalas]

I am using MB extension for a few days now alongside Adgurad extension.
I have unchecked the setting for ads blocking as recommended.
So far so good i just notice a fraction of slowdown in my browsing but i do have an old laptop.
Seems a good extension and for the moment i will keep it.

 

[toto_10]

I added Malwarebytes extension to Chrome alongside with uBlock Origin and I'm using Bitdefender TrafficLight. Is BTL unnecessary or should I keep it?