How do you know if an anti-virus test is any good?

[nishaddesilva]

Anti-virus or, as we say now in the industry, anti-malware testing has been around for years.

These tests and comparatives are the consumer reports of the IT security industry, aimed at educating both the anti-malware developer and the consumer on how a product performs.

There's been a fair bit of activity in the anti-malware testing world lately - both AV-Test and AV-Comparatives released major reports last week, and at Virus Bulletin we're putting the finishing touches to our latest comparative on Windows XP, due out in the next week or so.

As usual at this time of year I've been getting a lot of people asking me, why are they all different? How do I know who to believe? What makes one test better than another, or are they all equally brilliant/useless/biased/random?

They're never easy questions to answer.

Testing anti-malware products is a complex and difficult process, and 'reading' tests - judging their quality, significance and relevance to one's own personal requirements - can be equally taxing.

So, I thought it might help to put together some simple points about how to spot a quality test, and how to judge the relevance of its findings.

Source: http://nakedsecurity.sophos.com/2013/04/19/how-do-you-know-if-an-anti-virus-test-is-any-good/

 

[orestisssss]

The best thing to do is,
Run a VM and test each antivirus you want yourself with your settings and with live samples.
Only then can you be sure that you will get the protection you want.

 

[Deleted member 178]

Run a VM (...)

the first mistake , some products behave differently inside a VM.

 

[jamescv7]

The standards are really depends from the organizations and reviewer.

Caused the conduction of testing will generally conclude from the performance results.

So that make sure enough samples to proved its effectiveness, settings must be ordered prior for category and if the testing purpose design to updated or vulnerable.

Testing organizations are done actually doing pretty same methodology but varies on other complex type to make sure its a standard for certification.

 

[DrBeenGolfing]

The test samples are the thing.

 

[Gnosis]

60 known malicious .exe's
60 unknown malicious .exe's
60 malicious links

ON AN ACTUAL TEST MACHINE, not a VM.
End the test with HitMan Pro, and MBAM or MBAR.
That shall give a good idea.

I don't tend to like most security tests because of the default settings that are usually left in place. Certain tools can be tweaked where they are more aggressive, yet won't nag or freak out a noob, or end up with a slew of FP's. As far as expert users go, some of these suites can be made ridiculously better, and still not be to tedious after modifying the factory settings.

 

[Littlebits]

This best real-time AV test you can do:

1. Instal a real-time AV.
2. Follow the safety guidelines for downloading files and browsing the web.
3. Do not manually download samples to test it. Most samples available on malware download sites are not widespread (extremely rare), are sometimes out-dated (the life of malware is only about 90 days), some can no longer infect Windows because of recent updates and many samples are false positives. A good sample has to be verified active in still in the wild and widespread. Therefore it is impossible to get a good sample collection and an accurate results.
4. Utilize UAC, never allow unknown processes.
5. After using your own knowledge with the help of the real-time AV and it successfully protects you then it passes, if something gets past it then it fails.

Remember real-time AV's only help you that can not do all the work.
If you rely on any of them completely they will all fail you.

Too many users expect their AV to detect everything and that is not possible and never will be possible.

Therefore all AV tests are not accurate because all will fail to detect everything. Detection rate means nothing, if just one infection happens then the AV fails.

If you have been using basically the same AV for years and it has always did its job then there is no reason to change to another that might fail you just because some AV tests or user video reviews said it was the best.

So never fully trust any AV tests or video reviews, the situation might be very different on your system.

Thanks.