Real-World Antivirus Protection Test: Winners and Losers

Status
Not open for further replies.

Nico@FMA

Level 27
Thread author
Verified
May 11, 2013
1,687
363598-real-world-antivirus-protection-test-winners-and-losers.jpg

Your antivirus tool has many different opportunities to protect your PC from attack. It can totally block access to a malicious URL, wipe out the downloaded code on sight, recognize and prevent malicious behavior, even roll back the system changes made by a malware attack. It's easy to create a lab test that checks just one of these layers, for example, the ability to block malicious URLs or recognize malware by signature. Such a test is informative, but doesn't give the whole picture. AV-Comparatives runs a continual Real-World Protection test that lets each antivirus use all of its weapons against live malware. The latest summary of this test's results reveals a broad range of effectiveness.

Dynamic Testing
The full report goes into great detail about the exact test methodology. Briefly, researchers install 20 or more antivirus products on identical PCs. Every day they gather the latest malicious URLs and test whether each product protected the system. If the antivirus asks the user whether to block or allow any action, they always choose allow. An antivirus that successfully fends off compromise even when the user makes the wrong choice still gets full credit. If making the wrong choice leads to compromise, it gets half credit.

During May and June, the company's researchers ran over 4,000 such tests. They also checked for false positives—legitimate URLs or programs wrongly identified as malicious by the antivirus software. A product with more than the average number of false positives can lose points.

Winners
Nine products successfully protected against 99 percent or more of the samples. Eight of them, including Bitdefender and Kaspersky, earned the top rating, Advanced+. Due to false positives, F-Secure got knocked down one rank to Advanced.

Avast and Baidu made impressive comebacks. In the previous summary, both failed to achieve even the Standard rating. This time around Baidu rated Standard and Avast rose to Advanced.

Losers
AV-Comparatives ran this test under Windows 7, and included the optional Microsoft Security Essentials as a baseline. Had it been given an official rating, it wouldn't have reached the Standard level. Along with AhnLab and ThreatTrack VIPRE, Microsoft rated merely Tested.

McAfee, Trend Micro, and eScan all earned Advanced+ in the previous report. This time around they would have rated Advanced based solely on detection rate, but false positives dragged all three down to a Standard rating.

Also-Rans
Not every security vendor chooses to participate in testing by AV-Comparatives. G DATA's people don't approve of the way the testing system rates protection that depends on user interaction, so they've opted out. Symantec has long contended the file detection and retrospective tests performed by AV-Comparatives are irrelevant. Since those are included in the all-or-nothing package of tests, Symantec hasn't participated for years.

This time around, AV-Comparatives roped in Symantec and G DATA for testing, for informational purposes. The real-world test is exactly the kind of test Symantec believes should be universal, as it exercises the whole product. Symantec would have earned Advanced+ in this test, with a very high detection rate and no false positives. G DATA would have managed an Advanced rating.

Testing the actual effectiveness of antivirus products is a tough job. Doing it right requires dedication and creativity. It's not surprising that this particular test has won a number of awards from European governments and organizations. The testing experts at AV-Comparatives are doing a great job.

Hmm why am i not surprised about Symantec...
 

Nico@FMA

Level 27
Thread author
Verified
May 11, 2013
1,687
I still believe the best test for an anti-virus would be to let loose a bunch of 8 year olds hyped up on sugar on the computer, tell them they can do anything they want and then see how much malware ends up being blocked/left behind afterwards.

Well if they first would remove all those Asian Wakky Duda AV brands who use engines from others, Then they remove clones of others, and then they remove Av vendors that use other brands signatures.
Then they start testing products across the whole spectrum of capabilities on a LIVE computer both in clean state, infected state and in simulated state and then come up with the results. If they would only test real products.....
Then i would be really impressed.
 

Kate_L

in memoriam
Verified
Top Poster
Well-known
Jun 21, 2014
1,044
I don't trust AV-Test, AV-Comparaties ... I said it before and I will say it again ... a lot of engines are way better (Vipre, Baidu ...). And my logic is telling me that something is really wrong. How an AV that has BitDefender engine is worst then BitDefender, even if you didn't update in last 1-2 days should be equal or near. Also with Baidu ... it has Avira + Kaspersky and so on ...
 

conceptualclarity

Level 21
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Aug 23, 2013
1,072
http://www.av-comparatives.org/wp-content/uploads/2014/07/avc_prot_2014a_en.pdf

Panda free coming out on top is amazing. It's not the first time it's outdone most of the paid big boys, however. No longer can you talk of a "big three" in free anti-viruses (avira, avast!, and AVG) and leave out Panda and Qihoo.

It's good to see Emsisoft continuing to go strong. AVIRA pro did very well. Kaspersky dropping to fifth after winning the plurality of tests this year is a surprise. But it still had a 99.6% protection rate.

False positive champs are McAfee, TrendMicro, Vipre, and F-Secure.
 
  • Like
Reactions: Raider Red and marg

nissimezra

Level 25
Verified
Apr 3, 2014
1,460
People, please stick to the topic, and if you do not agree with someone, maintain your discussion according to topic name. If you have something personal to discuss, there is Private Message for this. Thanks.
stick to the topic? that's what they did! what is that all about? are you trying to prove yourself here as a stuff member?
 

Arakasi

Level 4
Verified
Jul 12, 2014
195
@nissimezra
He is staff, and its probable he has access to remove and delete posts where he sees fit.
Moderation.
Chances are he removed junk not related to topic and we can no longer see it??? ;)
 

nissimezra

Level 25
Verified
Apr 3, 2014
1,460
I don't trust AV-Test, AV-Comparaties ... I said it before and I will say it again ...

me to

i cant believe that panda is still on the top, i've tested it 5 times always killed my vm.
they probably using non zero day samples thats explain 88% mse when it doesn't get even near 50%
 
Last edited by a moderator:

TwinHeadedEagle

Level 41
Verified
Mar 8, 2013
22,627
stick to the topic? that's what they did! what is that all about? are you trying to prove yourself here as a stuff member?

I am trying to keep good members here, and to prevent bad behavior from the people who are too aggressive and to leave personal differences with someone out of this. If you cannot behave according to rules, then please avoid such discussions.
 

Exterminator

Community Manager
Verified
Staff Member
Well-known
Oct 23, 2012
12,527
The only "real world" test I am going to rely on is on my machines in my little part of the "real world".

every times i read their tests, it is "same old, same old"

next time they just have to copy-paste the report and change the date.

I agree 100%. I don't even read them anymore.I imagine it does spark some interesting debate,that is if everyone could stay on topic and play nice.

stick to the topic? that's what they did! what is that all about? are you trying to prove yourself here as a stuff member?

The chatbox type of replies and those that are offensive to other members have been deleted.You are not a staff member and cannot view them.TwinHeadedEagle has nothing to prove!!! Visit the malware removal section sometime for verification of my last statement. However he is a staff member with moderation duties,and deserves respect.If a member doesn't agree with something by all means PM us and we will be glad to explain our actions.Frankly,the calling out of staff & regular members in the forums has to come to an end.
 
Last edited:
G

Guest28

I dont believe this test at all from my own testings. If you know how to test an Anti-malware load up your vm with panda cloud and watch it epically fail.
 
Status
Not open for further replies.

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top