Disclaimer

This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
We encourage you to compare these results with others and take informed decisions on what security products to use.
Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

MacDefender

Level 3
Verified
A lot of the usual strong performers! I was surprised that F-Secure is way up there too with the top dogs. They pulled ahead of their base engine Avira too which seems to validate the notion of using several layered engines together. IMO it used to be nothing terribly special when they used Bitdefender but switching to Avira seemed to work out really well for them. Seems like Avira caught more of what their in house engines missed.


It was surprising to see Windows Defender perform a bit weaker than it does in other comparative tests.

Overall the FP rate seemed low for almost everyone which is good.
 

MacDefender

Level 3
Verified
Sophos Home Premium doing really well
In the past I was using Sophos UTM Home, their turn your PC into an enterprise firewall solution, and that came with a number of free Sophos endpoint protection licenses that are centrally managed by the Sophos firewall. I was fairly impressed with their protection and central management. Only problem was they switched since then to the XG Firewall and that product is a usability nightmare.
 

notabot

Level 15
In the past I was using Sophos UTM Home, their turn your PC into an enterprise firewall solution, and that came with a number of free Sophos endpoint protection licenses that are centrally managed by the Sophos firewall. I was fairly impressed with their protection and central management. Only problem was they switched since then to the XG Firewall and that product is a usability nightmare.
Last year they used to offer both UTM and XG free for home use but XG is where they were heading to. I've never used their UTM product (nor XG) for sure they didn't work together with Sophos Home Premium ( they did work together with the enterprise endpoints though as you say ).

If they add support for AMSI in SHP, I may make it my main product as I really like their web dashboard.
 

Andy Ful

Level 49
Verified
Trusted
Content Creator
SE Labs correctly awarded most of the tested AVs, because this test cannot differentiate between them due to statistical errors and small scoring differences.(y)
Like in other cases, only the average of several such tests can be meaningful, especially when the AV has got consistently high scores.
It is a waste of time to talk that Sophos has got a very good scoring (in one test), because it can be simply a statistical fluctuation. The best results in the year 2019 (3 SE Lab tests) has got Symantec Norton Security, and the worst G-Data IS. Webroot was tested only once this year. This test cannot properly measure the efficiency of Webroot's rollback feature, so there is no need to talk about Webroot here.
 
Last edited:

notabot

Level 15
SE Labs correctly awarded most of the tested AVs, because this test cannot differentiate between them due to statistical errors and small scoring differences.(y)
Like in other such tests, only the average of several such tests can be meaningful, especially when the AV has consistently high scores.
The top vs bottom AVs are fairly consistent, this doesn't mean this test is necessarily a good measure but if you look past & present test results, there is consistency of results over time
 

MacDefender

Level 3
Verified
Looks like they still have their UTM firewall for home

They sorta do but the 50 IP address limit is not ideal these days, between the explosion in IOT devices and with IPv6 how every device requests at least 4 different IPs for itself.

Plus UTM is basically dead end and no more active development. It's a shame -- UTM was at least understandable despite being clunky. XG is super pretty looking but the way that settings like IPS and web filtering attach to firewall flows is very confusing (not to mention it's got a totally useless form of QOS via unconditional bandwidth reservation)
But this is too off topic for this discussion! I don't have anything against Sophos but I do like some of the other top scorers here more!
 

Andy Ful

Level 49
Verified
Trusted
Content Creator
The problem with SE Labs tests (and generally with all real-wold tests) is that any popular AV (including Webroot) + Edge Chromium on Windows 10 will get very good scoring. Additionally, Webroot has an advantage of the rollback feature, even if it works on 50%. It is not stupid security for average users, until they can respect SmartScreen (anti-phishing and application reputation) in Edge Chromium.(y)
 

notabot

Level 15
The problem with SE Labs tests (and generally with all real-wold tests) is that any popular AV (including Webroot) + Edge Chromium on Windows 10 will get very good scoring. Additionally, Webroot has an advantage of the rollback feature, even if it works on 50%. It is not stupid security for average users, until they can respect SmartScreen (anti-phishing and application reputation) in Edge Chromium.(y)
Someone who's not ignoring SmartScreen, has good habits, uses mostly AppContainer'd apps & keeps their software up to date, barely needs an AV anyhow.

That said, I'd want testing to push suites to their limits, if a lot get 99%, the tester is not trying hard enough, also I'd want more granularity, on this test, what was signature block, what was behavioural, how was an exploit stopped/by which module and at what stage etc. - other labs do provide more details.
 

Andy Ful

Level 49
Verified
Trusted
Content Creator
Someone who's not ignoring SmartScreen, has good habits, uses mostly AppContainer'd apps & keeps their software up to date, barely needs an AV anyhow.

That said, I'd want testing to push suites to their limits, if a lot get 99%, the tester is not trying hard enough, also I'd want more granularity, on this test, what was signature block, what was behavioural, how was an exploit stopped/:unsure:by which module and at what stage etc. - other labs do provide more details.
You will be satisfied when looking at many tests on Malware Hub.:giggle:
 

notabot

Level 15
You will be satisfied when looking at many tests on Malware Hub.:giggle:
I'm not too familiar with the hub, I watch results from time to time, my understanding is that it evaluations suites vs raw samples which do not have motw so while the hub would put most modules of a suite to a harder test, it's not a real world scenario ( which the testing labs claim to do ), as it's already bypassing web filtering and cloud reputation (ie smartscreen).

Of course this means the other modules of a suite need to prove their worth at the hub, which may not necessarily happen during a lab's real world tests and this is very interesting, as these are the last line of defense and it's up to them to protect when the user manually bypasses smartscreen or when the malware delivery is done eg via exploit.
I've also seen custom configuration tested ( ie H_C ) in the hub which may not be what a testing lab is testing and those who do harden their OS's may find a testing lab's results irrelevant.

Are the hub top scorers different from testing labs top scorers?
 

Andy Ful

Level 49
Verified
Trusted
Content Creator
What is happnng with Webroot??? Their going down the tube in a hand basket!!!! Its sad to view these poor statics
If I correctly understood the information from their website, Webroot does not use cloud signatures, so the detection for fresh malware cannot be very good. The rollback feature can be an alternative solution to keep good protection. Furthermore, the vendor sells the Webroot SecureAnywhere Web Security Service in the business sector. As can be seen from the below list, Webroot has got a perfect score against targeted attacks and a poor score for web-downloads. The second can be simply covered by adding a web browser extension or using Chromium Edge.

ATTACK TYPES
Product (Web-Download, Targeted Attack)
Comodo Internet Security 75 25
F-Secure Safe 75 25
Kaspersky Internet Security 75 25
Sophos Home Premium 75 25
Symantec Norton Security 75 25
Trend Micro Internet Security 75 25
Check Point ZoneAlarm 74 25
ESET Internet Security 74 25
McAfee Internet Security 74 25
Microsoft Windows Defender 74 25
Avast Free Antivirus 73 25
AVG Antivirus Free Edition 73 25
Avira Free Security Suite 73 25
G-Data Internet Security 75 20
Webroot Antivirus 56 25

Edit.
The protection design of Webroot is not my favorite one. But, I think that we should not criticize it on the base of the test, which is not especially reliable to measure overall Webroot protection.
 
Last edited:

Andy Ful

Level 49
Verified
Trusted
Content Creator
I'm not too familiar with the hub, I watch results from time to time, my understanding is that it evaluations suites vs raw samples which do not have motw so while the hub would put most modules of a suite to a harder test, it's not a real world scenario ( which the testing labs claim to do ), as it's already bypassing web filtering and cloud reputation (ie smartscreen).

Of course this means the other modules of a suite need to prove their worth at the hub, which may not necessarily happen during a lab's real world tests and this is very interesting, as these are the last line of defense and it's up to them to protect when the user manually bypasses smartscreen or when the malware delivery is done eg via exploit.
I've also seen custom configuration tested ( ie H_C ) in the hub which may not be what a testing lab is testing and those who do harden their OS's may find a testing lab's results irrelevant.

Are the hub top scorers different from testing labs top scorers?
You probably see that you have somewhat contradictory wishes. If you will respect SmartScreen then you will not see behavior (and many exploit) blocks.:)
 

notabot

Level 15
You probably see that you have somewhat contradictory wishes. If you will respect SmartScreen then you will not see behavior (and many exploit) blocks.:)
I hope I won't, but legit and fully updated apps can be exploited by a zero day exploit and then a behavioural block becomes very important. Or an application may update and a supply chain attack may replace a dll with a malicious one, smartscreen won't catch that if the exe doesn't change.

The chance of something like this happening is small but to secure against these attack vectors, it's up to the BB.
 
Last edited: