Advice Request AV-Comparatives Hits "DIY Antivirus Testers" - Leave it to the professionals!

Please provide comments and solutions that are helpful to the author of this topic.

Ink

Administrator
Thread author
Verified
Staff Member
Well-known
Jan 8, 2011
22,361

Professional labs have the resources to use large amounts of relevant malware, and test all the security features of multiple antivirus programs simultaneously, with equal conditions for all the tested products. They can also investigate usability of the tested products, that is whether they create a lot of false alarms or slow your computer down in daily use.

Some of the test labs even undergo strict audits by ISO to proof, that they are independent, and work together with Universities for scientific research to keep the test methodologies state of the art.

Now, are you sure you want to rely on the results of DIY testing to protect your computer?
 

Like a Western!

Level 9
Verified
Well-known
Apr 6, 2016
440
AND you need to be sure that these are genuinely malicious
This, so true.
i remember that i had some samples from the Malware Hub here at MT, sent them over to the Dr.Web or F-Secure
and the files were not malicious in their opinion, the actual malware analysts. it doesn't mean if a file is on so called malware source websites it has to be malicious, but people just grab them and like they are sure it's a malware cause some AVs detecting it on VT which prove nothing, if you grab a file from somewhere like "app any run" or "virussign" and only 3-4 AV detecting it on VT, it doesn't mean you are a malware hunter or you just found a zero-day, sometimes it just ain't virus.
 

Like a Western!

Level 9
Verified
Well-known
Apr 6, 2016
440
Yeah, do we want to trust tests that are possibly extremely biased and are never shown taking place, or do we want to trust tests which are actually shown on video?
if you can't trust the independent organization you should not trust the tests that show you video too, for example when TPSC do a Bitdefender vs Kaspersky both free products, how can you trust that he is independent ? it's nothing personal it's just an example of the situation we got here.
how you can be sure that the the Youtube testers did not actually choose the samples on their package to let a special product win in the battle or another fail? that you cannot understand ever.

at least organization are more reliable and harder to buy than a youtuber etc...

HOW MUCH YOU NEED TO PAY TO BUY A YOUTUBER? maybe 100$? some keys?
how much they need to pay to get a decent result in independent orginazations like Av-Test or Av-Comparatives? best possible price is grands.
 

EndangeredPootis

Level 10
Verified
Well-known
Sep 8, 2019
461
if you can't trust the independent organization you should not trust the tests that show you video too, for example when TPSC do a Bitdefender vs Kaspersky both free products, how can you trust that he is independent ? it's nothing personal it's just an example of the situation we got here.
how you can be sure that the the Youtube testers did not actually choose the samples on their package to let a special product win in the battle or another fail? that you cannot understand ever.

at least organization are more reliable and harder to buy than a youtuber etc...

HOW MUCH YOU NEED TO PAY TO BUY A YOUTUBER? maybe 100$? some keys?
how much they need to pay to get a decent result in independent orginazations like Av-Test or Av-Comparatives? best possible price is grands.
I dont trust neither independent tester nor people who do it on video, if I want a test, ill do it myself, and keep it to myself, im simply stating the problem of bias when it comes to this kind of stuff, I did say it in a confusing (sarcastic sounding) way, but I dont agree with anyone
Take virusbulletin, all their antivirus they test always have a 100% detection ratio according to them, whats the point of their tests if all of them have a 100% detection ratio? do you really want to accept it as fact? why should you need to do other tests if all antiviruses have a 100% detection ratio? it just makes no sense, and when it comes to tests done on video, how do we know they havent been tampered with like you said?
 
Last edited:

Cortex

Level 26
Verified
Top Poster
Well-known
Aug 4, 2016
1,465
It does mention in the blurb to the effect of we can tell you if product slows your PC down, IMHO the only person who can do that is the owner of the PC you intend using it on as every PC hardware/software combination is different - Interesting read as recently I've started taking AV-Comparatives with 50kg of salt - I do read the info still but make up my own mind - Are they running scared?
 

mlnevese

Level 26
Verified
Top Poster
Well-known
May 3, 2015
1,531
Something I noticed in many YouTube tests is that when the tester executes undetected samples they are either perfectly legitimate applications or files that can't even be executed and many still count those as misses by whatever software they are testing.

If a security software is alerting you to something that can't even be executed, then it certainly has a problem...

I really doubt that a professional lab will use non-working or non-malicious software to test. Their credibility would suffer a serious hit if they did.
 

silversurfer

Level 85
Verified
Honorary Member
Top Poster
Content Creator
Malware Hunter
Well-known
Aug 17, 2014
10,048
This, so true.
i remember that i had some samples from the Malware Hub here at MT, sent them over to the Dr.Web or F-Secure
and the files were not malicious in their opinion, the actual malware analysts. it doesn't mean if a file is on so called malware source websites it has to be malicious, but people just grab them and like they are sure it's a malware cause some AVs detecting it on VT which prove nothing, if you grab a file from somewhere like "app any run" or "virussign" and only 3-4 AV detecting it on VT, it doesn't mean you are a malware hunter or you just found a zero-day, sometimes it just ain't virus.
You should like to mention that 99% of all samples from the Malware-Hub are for sure malicious/malware, claiming otherwise isn't really true!
 

TairikuOkami

Level 35
Verified
Top Poster
Content Creator
Well-known
May 13, 2017
2,452
Interesting read as recently I've started taking AV-Comparatives with 50kg of salt - I do read the info still but make up my own mind - Are they running scared?
I considered AV Comparatives somewhat trustworthy, but after this article, I will take them as seriously as a fake AV.

What is a fact: "A fact is a piece of information that is known to be consistent with objective reality and can be proven to be true with evidence."
Youtubers provide all the info, that can be easily repeated, AV comparative companies require you to trust them, that says it all and ... well go figure.
 

Vitali Ortzi

Level 22
Verified
Top Poster
Well-known
Dec 12, 2016
1,147
I considered AV Comparatives somewhat trustworthy, but after this article, I will take them as seriously as a fake AV.

What is a fact: "A fact is a piece of information that is known to be consistent with objective reality and can be proven to be true with evidence."
Youtubers provide all the info, that can be easily repeated, AV comparative companies require you to trust them, that says it all and ... well go figure.
Yeah!
That's insane 😱.

"usability of the tested products, that is whether they create a lot of false alarms or slow your computer down in daily use. "

Never found their testing to be good.
Especially in usability.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top