AV-Comparatives: Investigation in progress

[Enju]

"AV-C has uncovered an infringement of the testing agreement by one of the vendors participating in its tests. It has been found that a product submitted for testing by the vendor had been specifically engineered for the major testing labs, including AV-C; public availability of this version was limited. A second vendor is also being investigated for similar reasons. When this analysis is complete, AV-C will announce the measures it will take against the vendor(s) found to be in breach of contract."

There was a thread at kafan.cn about Tencent and (not 100% certain) Qihoo in this regard, but the post has since been deleted...
http://bbs.kafan.cn/thread-1824024-1-1.html

Update 04/28/2015:
"AV-Test, Virus Bulletin and AV-Comparatives had a conference call today. Further collaborative investigation is now in progress. We will give additional information in a joint statement with the other labs. We will keep you up to date."

Source: http://weblog.av-comparatives.org/?p=501

 

[FreddyFreeloader]

From Casablanca:

Captain Renault: I'm shocked, shocked to find that gambling is going on in here!

[a croupier hands Renault a pile of money]

Croupier: Your winnings, sir.

Captain Renault: Oh, thank you very much.

 

[Nirv5668]

Not surprising. I think it is likely companies design their products with testing labs in mind- which is not ideal. If they implement novel features that cannot be tested in the automated AV-comparatives environment, they lose out on potential PR and people wonder why they are not submitting for testing. So, the AV-comparatives environment, designed to be a convenience to AV-comparatives, becomes a limiting factor in AV software development and innovation. Yet, AV-comparatives has all the power here. All software companies would have to get together and agree not to design with AV-comparatives in mind...and that won't happen. Honestly, from my perspective it seems like AV-comparatives and others stifle innovation. At least these two companies didn't publicly release their testing lab specific version...I would guess other top scorers stay out of trouble by just selling everyone their testing lab engineered version .

This reminds me of the whole thing with Comodo...
https://www.melih.com/2011/11/27/av...-and-financial-deals-with-anti-virus-vendors/

Quote from AV-Comparatives:

About the whole product test, we would like to see you in, but, and this is an official statement, which you can post in your blog, your product with the current sandbox and the current way it is working, is not compatible with our automation system

 

[aztony]

Who is surprised by this? I would start with the fact that these AV companies have to pay AV-C, and the like, to participate in the testing of their products. This is taints the water for developers already under pressure for a good outcome for their respective products, plus the added pressure to get a positive return for their money/test fees. It is a strong inducement to do things, above and beyond, in order to get an edge on the competition. So what may have started out/intended as a well meaning public service venture becomes corrupt.

 

[FireShootSK]

And here is why is Avira first.

 

[woomera]

This reminds me of the whole thing with Comodo...
https://www.melih.com/2011/11/27/av...-and-financial-deals-with-anti-virus-vendors/

OMG never knew about this. shocking. changed my view about AVC, thanks for sharing.

 

[Ramona]

So people that wrote here about only few AV are "real" said the truth

 

[jamescv7]

Better late than never, finally they realize about these actions which should make proper and overhaul change the way of test.

In other side of picture, it might be a coincidence as the duty of AV is to collect everything all the information of files and analyze for checking however seems its an unfair, unjust and greedy to activate the program for testing purpose for obtaining high rate product.

Well again at the end of the day, user reviews are more accurate than testing organizations.

 

[aztony]

Well again at the end of the day, user reviews are more accurate than testing organizations.

+1. Although I pay attention to both, these days I put more stock in user reviews.

 

[Deleted member 21043]

At least now we know AV-C do try to be "fair" for each vendor and don't support cheating.

Although, the news is disapointing.

 

[Atlas147]

Very interesting, looking forward to seeing the results of who are the 2 AV vendors that were cheating in the test.

 

[omidomi]

And here is why is Avira first.

haha clever idea.
but AVira detection is good

 

[FireShootSK]

haha clever idea.
but AVira detection is good

of course, avira detection is great but no 100%

 

[omidomi]

of course, avira detection is great but no 100%

yep
100% is really funny, i said "good" not "great"
Good=lower 95
Great=Uper 95

 

[jogs]

If they really want to test any product then, they should get the products from the market instead of getting them from the vendors directly.

 

[Enju]

If they really want to test any product then, they should get hold of the products from the market.

AV Companies have to pay to get tested.

of course, avira detection is great but no 100%

They state on page 3: Even though some products reach 100% protection rates, doesn't mean that they always protect against every threat.

 

[Sr. Normal]

It still can complicate any more .
Another site that I like to look at is https://threatcenter.crdf.fr/?Stats
It manages some more " credible" percentages , however , I am impressed to see Badiu and Avast as above, there are other antivirus that I find most successful position.
The best thing is that you see it

 

[Ramona]

I think BitDefender and Panda are the two that cheat, we will see

 

[FireShootSK]

I think BitDefender and Panda are the two that cheat, we will see

BitDefender cheat on AV-Test.org i think and with panda i agree

 

[aztony]

I think BitDefender and Panda are the two that cheat, we will see

Not very likely. More probable is Qihoo and Tencent. There was at least one foreign media report posted by a Wilders member living in China, (article has since been taken down) that Tencent accused Qihoo with gaming the File detection AV-C test by providing a specifically designed version of the antivirus that was only for testing purposes, and not available to the general public. Qihoo then countered with a similar accusation against Tencent.