AV-Comparatives AV-Comparatives Real-World Protection Test Jul-Oct 2023

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

simmerskool

Level 38
Verified
Top Poster
Well-known
Apr 16, 2017
2,715
I think that's why many users use it. But I have to say that some of the recent additions in ESET have been questionable and IMO made the ESET experience much worse than before.
for me ESET SSP was a disaster. it borked my win10 hardware PC! and ESET tech support gave 4 conflicting answers depending on ESET tech of the day over a week period. But after 6 month hiatus I put NOD32 back on a VM running with Voodooshield -- it is light and @Shadowra gave this combo a strong rating.
 

Shadowra

Level 37
Verified
Top Poster
Content Creator
Malware Tester
Well-known
Sep 2, 2021
2,617
Avast beating Eset and Kaspersky is a shock.

I'm planning an Avast test at the end of November :)

And once again, no antivirus will protect you 100%.
Personally, I've chosen to use an enterprise NGAV solution. It's light, efficient and I'm not complaining ^^.

On the other hand, I wonder how Eset "tested" it……..
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,542
"bad" seems very relative by missing 1.8% or 9 of 513 IIRC with 0 false positives, while ALL of AV that did better also had very-bad false positives. Seems like there could be / is a strong negative correlation between compromised vs false positives, but don't know if this is causation evidence :unsure: Is that discussed by AV-C? I guess AV-C acknowledges some interaction as it does / can otherwise lower their AV rating depending on false positives. @Andy Ful any thoughts about this... :unsure:
Meanwhile ESET has 100% at AVLabs.pl :whistle: ...
The number of false positives can lower the AV Awards, but is ignored when calculating the AV protection score.
The testing methodology of tests made by AVLab was very different from the methodology used by AV-Test, AV-Comparatives, and SE-Labs. The results cannot be reliably compared.
 

simmerskool

Level 38
Verified
Top Poster
Well-known
Apr 16, 2017
2,715
The number of false positives can lower the AV Awards, but is ignored when calculating the AV protection score.
The testing methodology of tests made by AVLab was very different from the methodology used by AV-Test, AV-Comparatives, and SE-Labs. The results cannot be reliably compared.
by some correlation I meant (or thinking) the more sensitive the AV engine & its settings are, the more likely the AV will "detect" malware, but also more false positives too... skimming the chart, Avast did detect 100% but had 2 false positives, F-Secure 100% with 18 fp, Norton 99.8% w/ 17 fp, TrendMicro 100% w/ 35 fp. If ESET had a few fp, might then also have more valid detections, or is that concept apples & oranges :unsure: Eg perhaps ESET could tweak its code, catch 100% but with 5 or 10 fp... :unsure: I have in no way "analyzed" the type of malware, the 1.8% that ESET is missing... I quickly skimmed the AV-C report again, I'm not seeing eg the type of malware getting passed ESET. Sorry if I missed it, I need new eyeglasses and the first appointment is not until January.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,542
by some correlation I meant (or thinking) the more sensitive the AV engine & its settings are, the more likely the AV will "detect" malware, ...
Yes, I think so. This follows from the behavior-based detections, which have a threshold for malicious behavior. If you lower that threshold, then you get more detections (and more false positives too). In the case of Eset, it could easily increase the detection rate by enabling some HIPS, which are disabled in default settings.
 

blueblackwow65

Level 23
Verified
Well-known
Dec 19, 2012
1,253
I think above 98% is really the accurate average result of ESET. For simply protection alone, I think other tier 1 products like Kaspersky, Bitdefender would be a better choice in most cases. In AVC's real-world tests, ESET can not even get better result than the default Microsoft Defender in most occasions.
But ESET is very light and usually annoyance free, with no nags or popups. I think that's why many users use it. But I have to say that some of the recent additions in ESET have been questionable and IMO made the ESET experience much worse than before.

The products were tested in default settings and no, you don't need to change any ESET's settings. There are sensitivity settings in advance settings, but those are set to values that ESET thinks are appropriate for their products that would result in less false positives and better performance. ESET wouldn't put their product into tests if they didn't think the default settings weren't enough.
I don't agree nut that is your view no problem
 
  • Like
Reactions: brambedkar59

brambedkar59

Level 32
Verified
Top Poster
Well-known
Apr 16, 2017
2,115
Meanwhile ESET has 100% at AVLabs.pl :whistle: ...
I love their results, everybody gets 100% score, no winners no losers 🤣

I don't agree nut that is your view no problem
His view might be based on av-comparatives results for real world protection test. If you look back as far as 2019 (that's as far I got before I got bored), Eset has been consistently on the bottom portion of the results. I am not saying it's a bad AV, I like it, it's really light. But that's what av-comparatives report says make whatever you want of it.
 
Last edited:

Adrian Ścibor

From AVLab.pl
Verified
Well-known
Apr 9, 2018
214
Hello @jetman

I have such a table :). Keep in mind that not all products are listed.

View attachment 279721
Man, you are doing an excellent job with this table and an entire summary, keep it up :)

I love their results, everybody gets 100% score, no winners no losers 🤣
You had in mind we cheat the users? Keep in mind we must comply to hard technical details and rules of AMTSO, everything has to be transparent in the background between AMTSO, Vendors and labs. If you wish, we can share a database dump of the test, so you can check the results yourself. We use in the test a random, unique files (by SHA256) and real live URLs with potential malicious files. Currently, some of them (7-10%) is a 0-day file (according to the mks vir engine - see at the CSV malware table and search "RoundKick", it means non-signature detection). "Based on several recent editions of security tests, we have come to the conclusion that solutions with multi-layered protection are the key to achieving the best results in protection against malware." - you can read in the newest summary.

This is a discussion of AV-C result. They have different methodology, evaluation, samples used in the test and everything else in the background.

If you wish, please consider to contact with us. We can include some of your malware set into the test (you have to use a URL to share [IP / domain]) and compare the result with included vendors. Only you and we will know about these SHA256s, so we can disclose these results to the MalwareTips community at the end. Of course, such a sample has to go through several stages to qualify to the test, but it is just like any other sample (have a look at an enclosed image).
 

Attachments

  • malware-selecting (1).jpg
    malware-selecting (1).jpg
    112.1 KB · Views: 123

brambedkar59

Level 32
Verified
Top Poster
Well-known
Apr 16, 2017
2,115
You had in mind we cheat the users?
Not at all, I have no reason to believe that. I saw the latest test results (Sept 2023) on your website & just found it funny, like what are the odds that all 15 products tested gave 100% detection result out of 343 malwares tested. I was in no way suggesting that you intentionally faked/cheated the results. I saw older test results and saw that products do actually fail often, September 2023 test was the only one where every product got 100%, last time this happened was in Nov 2022.

If you wish, we can share a database dump of the test, so you can check the results yourself. We use in the test a random, unique files (by SHA256) and real live URLs with potential malicious files. Currently, some of them (7-10%) is a 0-day file (according to the mks vir engine - see at the CSV malware table and search "RoundKick", it means non-signature detection). "Based on several recent editions of security tests, we have come to the conclusion that solutions with multi-layered protection are the key to achieving the best results in protection against malware." - you can read in the newest summary.
That would be pointless & unnecessary cause I am not questioning your methodology, that would be stupid of me because it's not my field of expertise. That would be like you questioning me about my RCC beam design.

This is a discussion of AV-C result. They have different methodology, evaluation, samples used in the test and everything else in the background.
I was replying to a member who posted results from avlab and my other comment was to a different member about AV-C. It was a not a comparison b/w these two results.

My post about avlab was only for light humor, I joke a lot if you saw me on the forum before. I apologize if my post seemed to imply anything bad about avlab, that was not my intention. I appreciate developers/officials from other organizations/groups interacting here with us on MT.

Cheers!
 

Virtuoso

Level 3
Well-known
Feb 21, 2022
109
F-Secure having so many false positives is not a surprise for me, many times I have to disable its extension to browse perfectly safe websites, heck once F-Secure blocked a payment gateway. The only reason I am continuing with it is because I got free four year license here.
 
  • Wow
Reactions: simmerskool

Jonny Quest

Level 22
Verified
Top Poster
Well-known
Mar 2, 2023
1,132
F-Secure having so many false positives is not a surprise for me, many times I have to disable its extension to browse perfectly safe websites, heck once F-Secure blocked a payment gateway. The only reason I am continuing with it is because I got free four year license here.

Over the years it's always had that issue as far as their testing, not sure why they, F-Secure, can't resolve it. Just from my experience and not trying to contradict what you're saying, but I can't recall a website FP yet. I've only been running it consistently for about 7 months on 2-3 PCs using Chrome and now Brave without any problems. But I'm not surfing all over the web, just to my favorite sites, so that may be a reason? In fact, this morning in checking from my F-Secure portal, Subscription, was shown a deal that I couldn't resist, so I added another year to my subscription :)
 

simmerskool

Level 38
Verified
Top Poster
Well-known
Apr 16, 2017
2,715
In fact, this morning in checking from my F-Secure portal, Subscription, was shown a deal that I couldn't resist, so I added another year to my subscription :)
I did not have many fp with F-Secure, but when my subscription ended, FS didn't show me any deals, seemed more like they wanted me to pay triple... :unsure: FS offered you a deal or some 3d party??
 
  • Like
Reactions: Nevi and roger_m

Jonny Quest

Level 22
Verified
Top Poster
Well-known
Mar 2, 2023
1,132
I did not have many fp with F-Secure, but when my subscription ended, FS didn't show me any deals, seemed more like they wanted me to pay triple... :unsure: FS offered you a deal or some 3d party??
What I did was check from my portal, and the price seemed really good, so I phoned support to confirm the price as well as if it would automatically be added to my current subscription, and extend it. He said as long as you link through your portal, it will merge the subscription, compared to going online to the renewal page? He also said that the link from the portal would also "sync" and be more in line with the deal I originally got with my initial purchase and the number of devices, in my case 7. When I hit the drop-down for the number of devices was surprised by the amount of device options I was given. It came down to $87.04 for 7 devices for 1 year (2 years I would have saved $40. over that period) and when I got the PayPal receipt, there was a bit of a discount to $67.34. I also like that the auto-renewal option to disable is plain as day right under the amount

I asked if I should go for the 2 years for the better deal, and he mentioned the prices could go up a bit during the year, but I would still get a good deal. Maybe these prices were also due to the time of the year, Black Friday type of thing?
 
Last edited:

simmerskool

Level 38
Verified
Top Poster
Well-known
Apr 16, 2017
2,715
What I did was check from my portal, and the price seemed really good, so I phoned support to confirm the price as well as if it would automatically be added to my current subscription, and extend it. He said as long as you link through your portal, it will merge the subscription, compared to going online to the renewal page? He also said that the link from the portal would also "sync" and be more in line with the deal I originally got with my initial purchase and the number of devices, in my case 7. When I hit the drop-down for the number of devices was surprised by the amount of device options I was given. It came down to $87.04 for 7 devices for 1 year (2 years I would have saved $40. over that period) and when I got the PayPal receipt, there was a bit of a discount to $67.34. I also like that the auto-renewal option to disable is plain as day right under the amount

I asked if I should go for the 2 years for the better deal, and he mentioned the prices could go up a bit during the year, but I would still get a good deal. Maybe these prices were also due to the time of the year, Black Friday type of thing?
To me it seems like the FS distinction between renewal online and going thru your portal is an oddly fine line. :unsure: I'm 98% sure I was logged into my.account FS portal, but maybe not. I liked F-Secure AV (Total) and would have purchased another license from FS for about the same price as my original license which I had bought from FS. I "soured" on this F-Secure / WithSecure renewal business model. Good to know you got a good or better deal. I will probably look at FS again when one of my other licenses expires.
 
F

ForgottenSeer 103564

Default settings are for balanced protection and usability. The later part being observed and important. Advanced settings exist for reason in the more advanced security products. Not only is eset not even close to locked down at default settings but custom rules can be placed yet.
Just like other more advanced products it takes knowledge and experience to gain the most from it in levels of protection. I can not stress that enough.
 

Jonny Quest

Level 22
Verified
Top Poster
Well-known
Mar 2, 2023
1,132
To me it seems like the FS distinction between renewal online and going thru your portal is an oddly fine line. :unsure: I'm 98% sure I was logged into my.account FS portal, but maybe not. I liked F-Secure AV (Total) and would have purchased another license from FS for about the same price as my original license which I had bought from FS. I "soured" on this F-Secure / WithSecure renewal business model. Good to know you got a good or better deal. I will probably look at FS again when one of my other licenses expires.

I was in evaluation mode last night of where I was at with my subscriptions, $$$ (VPN, PM and AV) in trying to narrow it down and simplify things. I'll let my Nord subscription expire, and after my 3-month trial of Mullvad is over I'll let that go (great VPN but costly). Some of the AVs want you to upgrade to the next tier of product if you need something more than 5 devices. F-Secure's VPN works great with our ISP, as you know, and the Password Vault, although being a bit spartan, is very usable for Chrome, FF and Edge. So F-Secure would cover all those bases and as needed to use my 1Password subscription for Brave unless I eventually switch to Bitwarden. But with the deal I got with F-Secure, wouldn't mind renewing 1Password when that time comes.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top