Here's why i strongly disagree with the latest AV-Test results

Do you agree with the latest results of AV-Test ?

  • Yes, they know what they are doing.

    Votes: 5 9.4%
  • I have some doubts, but i somewhat believe them.

    Votes: 28 52.8%
  • I disagree with their results.

    Votes: 8 15.1%
  • I don't trust any independent testing lab anyway.

    Votes: 12 22.6%

  • Total voters
    53
M

MSDOS8bit

Thread author
So as many of you, i tend to disagree with some professional antivirus testing such as AV-Test or AV-Comparatives. I might find that a security product is over rated or disagree with how the tests are made. For instance, Emsisoft happens to be among the last on AV-Comparatives quite often because they consider user-dependent actions the same as a miss. I disagree with these results as i know Emsisoft is one of the best product out there. Still i understand how they arrived to that conclusion.

But the latest results on AV-Test are a complete joke as you can see here. According to them, if you select by protection, Eset and Avira are behind Microsoft Security Essentials. I'm not saying Microsoft's protection is garbage, as i know they have improved lately, but there is no way that in real life condition Security Essentials or Defender offers better protection than Eset or Avira.

Also if you look at another one of their test from October 2016 (their latest Windows 10 test), AhnLab Internet Security ranked the highest. While i'm not familiar with this company, i highly doubt they provide better protection than Bitdefender, Kaspersky, Norton, Trend Micro and F-Secure.

So what are your opinions on these results ? Do you agree with those or no ? And why ?

Thanks for taking the time to read me ;)
 

Arequire

Level 29
Verified
Top Poster
Content Creator
Feb 10, 2017
1,815
I don't tend to pay too much attention to professional testing labs and their results. Their results are only indicative of the exact samples used and I find their usage of percentages as their scoring mechanism completely senseless.

While Microsoft has vastly improved MSE/WD's protection over the years their dynamic detection is still sorely lacking and I would be hesitant to say anyone use it without supplemental protection.
 
5

509322

Thread author
Lab tests are just an indicator of protections at default settings. Results will vary - sometimes significantly - over time. The most accurate assessment is to consider how a product performs and protects at the various labs over a long period of time.

Lab test results are just a single piece of the puzzle. For example, you might install the top 5 scoring security software on your system only to find out that none of them are to your liking for a whole host of potential reasons.

It's important to read the lab test reports in detail - especially the methodology. Very few people bother.

While the labs try to standardize their tests and make the testing consistent, testing is not perfect, not comprehensive, and it is not without valid criticisms\concerns.
 

Amelith Nargothrond

Level 12
Verified
Top Poster
Well-known
Mar 22, 2017
587
I don't particularly care about these results (but i take notice of the results nevertheless). There are some products we all know are in the top 10, because of several reviews (independent labs, independents reviewers), because of our experience with them, and there are advertised products ("as seen on tv" kind of products).

There is no perfect AV. The perfect AV is the one it fits your needs the most. In the top of these lists, all are pretty good and offer good protection. It's the lower end you should consider, as even these labs can't afford to mark a product with one star if it's really not close to one start. Maybe even the "middle class" is worth analysing. On the top, differences are very little, it's all about marketing and support, these could make a significant difference.

Update: there is a thread on MT about Panda 360. The guy advertising the product showed some lab results. The lab said this AV blocked 100% of 0-day malware. I took a closer look of their product. Turned out to be an app firewall with whitelisting features. Many of the samples were not identified as malware but blocked by the firewall, and the results were interpreted as: the AV blocked these because this time this really is the true next-gen AV. It's not the AV itself that blocked the malware, the samples were not identified as malware, not even close, the firewall feature from the software suite blocked them. So it really depends on the perception of the user and the test itself.
 
Last edited:
D

Deleted member 178

Thread author
But the latest results on AV-Test are a complete joke as you can see here. According to them, if you select by protection, Eset and Avira are behind Microsoft Security Essentials. I'm not saying Microsoft's protection is garbage, as i know they have improved lately, but there is no way that in real life condition Security Essentials or Defender offers better protection than Eset or Avira.

And once again people don't read the test methodology.

AV-TEST – The Independent IT-Security Institute

This test is all about Widespread/Prevalence , and we all know that MS is the best security apps about prevalence , its database is unmatched. Now, if it was focused on 0-day test , it would fall behind.

People , read before stating...

Anyway , Test Labs are just a representation of a given data in a given moment. So useless.
 
Last edited by a moderator:

Xsjx

Level 13
Verified
Feb 21, 2017
613
Lets clearly say that this is not true..

Even here in the malware vault i sometimes think this isnt the true score

You should not test it with random samples, You need to test Daily things that can happen...

Also last the more $ you give the higher the ranking.

Also i think ts 360 is not kicked out because of not fair settings i think its more about that Bitdefender,Kaspersky paid to let some people think its a fake av or like some of the worst av..

And yes they could include it with worst results but Hey who has believed it?


Actually like 22 % believe it right now that Defender is better than Avira,Eset.

I should say Just use your millitary grade 'Defender'
 
D

Deleted Member 3a5v73x

Thread author
Also i think ts 360 is not kicked out because of not fair settings i think its more about that Bitdefender,Kaspersky paid to let some people think its a fake av or like some of the worst av..
Do you believe that big companies would do such shady methods to eliminate/impact other competitors and risk with their own reputation? Knowing that Bitdefenders engine is used in many AV vendors now, I highly doubt it. o_O
 
  • Like
Reactions: frogboy and JB007

Xsjx

Level 13
Verified
Feb 21, 2017
613
Do you believe that big companies would do such shady methods to eliminate/impact other competitors and risk with their own reputation? Knowing that Bitdefenders engine is used in many AV vendors now, I highly doubt it. o_O

Intel did it with AMD so yes i think Kaspersky and bitdefender could or does it too.
 

ZeroDay

Level 30
Verified
Top Poster
Well-known
Aug 17, 2013
1,905
I think Lav test a good indicator of protection at that given moment, and, as said above looking for products that consistently perform well over a long period is a good way of evaluating a product's efficiency.

Look at Kaspersky, They've been in the top 3 for a very long time, Kaspersky also performs well in the Malware hub on a daily basis. Then if you look at AV-Test results for Comodo it's detection rate isn't the best but it's zero day protection always does extremely well, which is also reflected in the malware hub. Same with Avira, when it comes to just on demand scanning it's signatures have proved to be among the very best for a long time. I'm just using the above products as examples. I believe that if we're going to judge a products based on tests we need to look at it from multiple different angles to get the whole picture, or atleast as close to it as we can.
 
M

MSDOS8bit

Thread author
Lab tests are just an indicator of protections at default settings. Results will vary - sometimes significantly - over time. The most accurate assessment is to consider how a product performs and protects at the various labs over a long period of time.

Lab test results are just a single piece of the puzzle. For example, you might install the top 5 scoring security software on your system only to find out that none of them are to your liking for a whole host of potential reasons.

It's important to read the lab test reports in detail - especially the methodology. Very few people bother.

While the labs try to standardize their tests and make the testing consistent, testing is not perfect, not comprehensive, and it is not without valid criticisms\concerns.

Thanks a lot for this comprehensive and detailed info, i now know more about how tests works. Also it's hard to argue with a developer from AppGuard ;)
 
Last edited by a moderator:
M

MSDOS8bit

Thread author
And once again people don't read the test methodology.

AV-TEST – The Independent IT-Security Institute

This test is all about Widespread/Prevalence , and we all know that MS is the best security apps about prevalence , its database is unmatched. Now, if it was focused on 0-day test , it would fall behind.

People , read before stating...

Anyway , Test Labs are just a representation of a given data in a given moment. So useless.

I understand what you mean, and while it's true i didn't pay attention to how tests where done, i was just doubtful of the results. Thanks for your input that allowed me to understand more to it ;)
 
M

MSDOS8bit

Thread author
I think Lav test a good indicator of protection at that given moment, and, as said above looking for products that consistently perform well over a long period is a good way of evaluating a product's efficiency.

Look at Kaspersky, They've been in the top 3 for a very long time, Kaspersky also performs well in the Malware hub on a daily basis. Then if you look at AV-Test results for Comodo it's detection rate isn't the best but it's zero day protection always does extremely well, which is also reflected in the malware hub. Same with Avira, when it comes to just on demand scanning it's signatures have proved to be among the very best for a long time. I'm just using the above products as examples. I believe that if we're going to judge a products based on tests we need to look at it from multiple different angles to get the whole picture, or atleast as close to it as we can.

Yes, i agree that what matters are indeed the results in the long run. And i understand that AV-Test are a respectable company. Still, to see Eset and Avira beat by Microsoft Security Essentials, whatever the methodology of the tests where, is like looking at a car test saying that a Toyota family van has surpassed a Ferrari and a Mercedes in term of speed. I understand now there's more to it, but still that was kind of hard to believe (and still is). Anyway thanks for taking the time to answer me ;)
 
  • Like
Reactions: ZeroDay

Ink

Administrator
Verified
Staff Member
Well-known
Jan 8, 2011
22,361
Everybody knows Microsoft scores top in Prevalence tests, as was reported last year.

When the people's favourite Antivirus scores below another product they think is mediocre, there's no reason for them to assume the tests are flawed. That's why I use software that works for me and not give a damn what others say "why you using that PoC software...". :)

Don't forget that no words are spoken when Microsoft scores the least points.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top