AV-Comparatives AV-Comparatives Real-World Protection Test Jul-Oct 2023

[simmerskool]

I think that's why many users use it. But I have to say that some of the recent additions in ESET have been questionable and IMO made the ESET experience much worse than before.

for me ESET SSP was a disaster. it borked my win10 hardware PC! and ESET tech support gave 4 conflicting answers depending on ESET tech of the day over a week period. But after 6 month hiatus I put NOD32 back on a VM running with Voodooshield -- it is light and @Shadowra gave this combo a strong rating.

 

[Jengo]

Avast beating Eset and Kaspersky is a shock.

Avast now a days really one of the best AV.

 

[Shadowra]

Avast beating Eset and Kaspersky is a shock.

I'm planning an Avast test at the end of November

And once again, no antivirus will protect you 100%.
Personally, I've chosen to use an enterprise NGAV solution. It's light, efficient and I'm not complaining ^^.

On the other hand, I wonder how Eset "tested" it……..

 

[comolokko]

I'm planning an Avast test at the end of November

And once again, no antivirus will protect you 100%.
Personally, I've chosen to use an enterprise NGAV solution. It's light, efficient and I'm not complaining ^^.

On the other hand, I wonder how Eset "tested" it……..

I'm looking forward to it.

 

[Andy Ful]

"bad" seems very relative by missing 1.8% or 9 of 513 IIRC with 0 false positives, while ALL of AV that did better also had very-bad false positives. Seems like there could be / is a strong negative correlation between compromised vs false positives, but don't know if this is causation evidence Is that discussed by AV-C? I guess AV-C acknowledges some interaction as it does / can otherwise lower their AV rating depending on false positives. @Andy Ful any thoughts about this...
Meanwhile ESET has 100% at AVLabs.pl ...

The number of false positives can lower the AV Awards, but is ignored when calculating the AV protection score.
The testing methodology of tests made by AVLab was very different from the methodology used by AV-Test, AV-Comparatives, and SE-Labs. The results cannot be reliably compared.

 

[simmerskool]

The number of false positives can lower the AV Awards, but is ignored when calculating the AV protection score.
The testing methodology of tests made by AVLab was very different from the methodology used by AV-Test, AV-Comparatives, and SE-Labs. The results cannot be reliably compared.

by some correlation I meant (or thinking) the more sensitive the AV engine & its settings are, the more likely the AV will "detect" malware, but also more false positives too... skimming the chart, Avast did detect 100% but had 2 false positives, F-Secure 100% with 18 fp, Norton 99.8% w/ 17 fp, TrendMicro 100% w/ 35 fp. If ESET had a few fp, might then also have more valid detections, or is that concept apples & oranges Eg perhaps ESET could tweak its code, catch 100% but with 5 or 10 fp... I have in no way "analyzed" the type of malware, the 1.8% that ESET is missing... I quickly skimmed the AV-C report again, I'm not seeing eg the type of malware getting passed ESET. Sorry if I missed it, I need new eyeglasses and the first appointment is not until January.

 

[Andy Ful]

by some correlation I meant (or thinking) the more sensitive the AV engine & its settings are, the more likely the AV will "detect" malware, ...

Yes, I think so. This follows from the behavior-based detections, which have a threshold for malicious behavior. If you lower that threshold, then you get more detections (and more false positives too). In the case of Eset, it could easily increase the detection rate by enabling some HIPS, which are disabled in default settings.

 

[blueblackwow65]

I think above 98% is really the accurate average result of ESET. For simply protection alone, I think other tier 1 products like Kaspersky, Bitdefender would be a better choice in most cases. In AVC's real-world tests, ESET can not even get better result than the default Microsoft Defender in most occasions.
But ESET is very light and usually annoyance free, with no nags or popups. I think that's why many users use it. But I have to say that some of the recent additions in ESET have been questionable and IMO made the ESET experience much worse than before.

The products were tested in default settings and no, you don't need to change any ESET's settings. There are sensitivity settings in advance settings, but those are set to values that ESET thinks are appropriate for their products that would result in less false positives and better performance. ESET wouldn't put their product into tests if they didn't think the default settings weren't enough.

I don't agree nut that is your view no problem

 

[brambedkar59]

Meanwhile ESET has 100% at AVLabs.pl ...

I love their results, everybody gets 100% score, no winners no losers

I don't agree nut that is your view no problem

His view might be based on av-comparatives results for real world protection test. If you look back as far as 2019 (that's as far I got before I got bored), Eset has been consistently on the bottom portion of the results. I am not saying it's a bad AV, I like it, it's really light. But that's what av-comparatives report says make whatever you want of it.

 

[Adrian Ścibor]

Hello @jetman

I have such a table . Keep in mind that not all products are listed.

View attachment 279721

Man, you are doing an excellent job with this table and an entire summary, keep it up

I love their results, everybody gets 100% score, no winners no losers

You had in mind we cheat the users? Keep in mind we must comply to hard technical details and rules of AMTSO, everything has to be transparent in the background between AMTSO, Vendors and labs. If you wish, we can share a database dump of the test, so you can check the results yourself. We use in the test a random, unique files (by SHA256) and real live URLs with potential malicious files. Currently, some of them (7-10%) is a 0-day file (according to the mks vir engine - see at the CSV malware table and search "RoundKick", it means non-signature detection). "Based on several recent editions of security tests, we have come to the conclusion that solutions with multi-layered protection are the key to achieving the best results in protection against malware." - you can read in the newest summary.

This is a discussion of AV-C result. They have different methodology, evaluation, samples used in the test and everything else in the background.

If you wish, please consider to contact with us. We can include some of your malware set into the test (you have to use a URL to share [IP / domain]) and compare the result with included vendors. Only you and we will know about these SHA256s, so we can disclose these results to the MalwareTips community at the end. Of course, such a sample has to go through several stages to qualify to the test, but it is just like any other sample (have a look at an enclosed image).

 

[Jengo]

I love their results, everybody gets 100% score, no winners no losers


i also noticed this with AVLab.pl, bit the same as AV-Test, we must keep everybody happy, no winners and no loosers ( just joking )

 

[brambedkar59]

You had in mind we cheat the users?

Not at all, I have no reason to believe that. I saw the latest test results (Sept 2023) on your website & just found it funny, like what are the odds that all 15 products tested gave 100% detection result out of 343 malwares tested. I was in no way suggesting that you intentionally faked/cheated the results. I saw older test results and saw that products do actually fail often, September 2023 test was the only one where every product got 100%, last time this happened was in Nov 2022.

If you wish, we can share a database dump of the test, so you can check the results yourself. We use in the test a random, unique files (by SHA256) and real live URLs with potential malicious files. Currently, some of them (7-10%) is a 0-day file (according to the mks vir engine - see at the CSV malware table and search "RoundKick", it means non-signature detection). "Based on several recent editions of security tests, we have come to the conclusion that solutions with multi-layered protection are the key to achieving the best results in protection against malware." - you can read in the newest summary.

That would be pointless & unnecessary cause I am not questioning your methodology, that would be stupid of me because it's not my field of expertise. That would be like you questioning me about my RCC beam design.

This is a discussion of AV-C result. They have different methodology, evaluation, samples used in the test and everything else in the background.

I was replying to a member who posted results from avlab and my other comment was to a different member about AV-C. It was a not a comparison b/w these two results.

My post about avlab was only for light humor, I joke a lot if you saw me on the forum before. I apologize if my post seemed to imply anything bad about avlab, that was not my intention. I appreciate developers/officials from other organizations/groups interacting here with us on MT.

Cheers!

 

[Virtuoso]

F-Secure having so many false positives is not a surprise for me, many times I have to disable its extension to browse perfectly safe websites, heck once F-Secure blocked a payment gateway. The only reason I am continuing with it is because I got free four year license here.

 

[tech mantou]

F-Secure having so many false positives is not a surprise for me, many times I have to disable its extension to browse perfectly safe websites, heck once F-Secure blocked a payment gateway. The only reason I am continuing with it is because I got free four year license here.

false positives are blown far out of proportion

no big deal

 

[Jonny Quest]

F-Secure having so many false positives is not a surprise for me, many times I have to disable its extension to browse perfectly safe websites, heck once F-Secure blocked a payment gateway. The only reason I am continuing with it is because I got free four year license here.

Over the years it's always had that issue as far as their testing, not sure why they, F-Secure, can't resolve it. Just from my experience and not trying to contradict what you're saying, but I can't recall a website FP yet. I've only been running it consistently for about 7 months on 2-3 PCs using Chrome and now Brave without any problems. But I'm not surfing all over the web, just to my favorite sites, so that may be a reason? In fact, this morning in checking from my F-Secure portal, Subscription, was shown a deal that I couldn't resist, so I added another year to my subscription

 

[simmerskool]

In fact, this morning in checking from my F-Secure portal, Subscription, was shown a deal that I couldn't resist, so I added another year to my subscription

I did not have many fp with F-Secure, but when my subscription ended, FS didn't show me any deals, seemed more like they wanted me to pay triple... FS offered you a deal or some 3d party??

 

[Jonny Quest]

I did not have many fp with F-Secure, but when my subscription ended, FS didn't show me any deals, seemed more like they wanted me to pay triple... FS offered you a deal or some 3d party??

What I did was check from my portal, and the price seemed really good, so I phoned support to confirm the price as well as if it would automatically be added to my current subscription, and extend it. He said as long as you link through your portal, it will merge the subscription, compared to going online to the renewal page? He also said that the link from the portal would also "sync" and be more in line with the deal I originally got with my initial purchase and the number of devices, in my case 7. When I hit the drop-down for the number of devices was surprised by the amount of device options I was given. It came down to $87.04 for 7 devices for 1 year (2 years I would have saved $40. over that period) and when I got the PayPal receipt, there was a bit of a discount to $67.34. I also like that the auto-renewal option to disable is plain as day right under the amount

I asked if I should go for the 2 years for the better deal, and he mentioned the prices could go up a bit during the year, but I would still get a good deal. Maybe these prices were also due to the time of the year, Black Friday type of thing?

 

[simmerskool]

What I did was check from my portal, and the price seemed really good, so I phoned support to confirm the price as well as if it would automatically be added to my current subscription, and extend it. He said as long as you link through your portal, it will merge the subscription, compared to going online to the renewal page? He also said that the link from the portal would also "sync" and be more in line with the deal I originally got with my initial purchase and the number of devices, in my case 7. When I hit the drop-down for the number of devices was surprised by the amount of device options I was given. It came down to $87.04 for 7 devices for 1 year (2 years I would have saved $40. over that period) and when I got the PayPal receipt, there was a bit of a discount to $67.34. I also like that the auto-renewal option to disable is plain as day right under the amount

I asked if I should go for the 2 years for the better deal, and he mentioned the prices could go up a bit during the year, but I would still get a good deal. Maybe these prices were also due to the time of the year, Black Friday type of thing?

To me it seems like the FS distinction between renewal online and going thru your portal is an oddly fine line. I'm 98% sure I was logged into my.account FS portal, but maybe not. I liked F-Secure AV (Total) and would have purchased another license from FS for about the same price as my original license which I had bought from FS. I "soured" on this F-Secure / WithSecure renewal business model. Good to know you got a good or better deal. I will probably look at FS again when one of my other licenses expires.

 

[ForgottenSeer 103564]

Default settings are for balanced protection and usability. The later part being observed and important. Advanced settings exist for reason in the more advanced security products. Not only is eset not even close to locked down at default settings but custom rules can be placed yet.
Just like other more advanced products it takes knowledge and experience to gain the most from it in levels of protection. I can not stress that enough.

 

[Jonny Quest]

To me it seems like the FS distinction between renewal online and going thru your portal is an oddly fine line. I'm 98% sure I was logged into my.account FS portal, but maybe not. I liked F-Secure AV (Total) and would have purchased another license from FS for about the same price as my original license which I had bought from FS. I "soured" on this F-Secure / WithSecure renewal business model. Good to know you got a good or better deal. I will probably look at FS again when one of my other licenses expires.

I was in evaluation mode last night of where I was at with my subscriptions, $$$ (VPN, PM and AV) in trying to narrow it down and simplify things. I'll let my Nord subscription expire, and after my 3-month trial of Mullvad is over I'll let that go (great VPN but costly). Some of the AVs want you to upgrade to the next tier of product if you need something more than 5 devices. F-Secure's VPN works great with our ISP, as you know, and the Password Vault, although being a bit spartan, is very usable for Chrome, FF and Edge. So F-Secure would cover all those bases and as needed to use my 1Password subscription for Brave unless I eventually switch to Bitwarden. But with the deal I got with F-Secure, wouldn't mind renewing 1Password when that time comes.