Qihoo and testing labs: Multiple engine technology explained

Status
Not open for further replies.

hoang2007

Level 1
Thread author
Verified
Feb 22, 2015
23
April 30th, Qihoo 360 received comments from its industry partners with allegation of inappropriate behaviour on the benchmarking processes in test labs. We regret that this behaviour has resulted into such comments from these labs, who we recognize as reference for security benchmarking. However, we hereby offer our perspective to the alleged comments.

The allegation highlights that the default configuration of the product available for the public, differs from the configuration used by the labs for testing. This configuration was explicitly declared upon submission of the tests, and was thereafter confirmed by the test labs.

In the public version, 3rd party engine is off by default, in the consideration that the majority of our users, are running on lower computing power. To satisfy lab conditions, the consideration of power constraint was therefore discarded. In any case, no alleged comment indicates that the level of protection from the product, is lower than the records achieved during the testing sessions.

Qihoo is committed to provide free security solutions, in order to convert security a commodity for all PC users. In addition, our product offers multiple engines, and regardless of the initial configuration, it is the user who has the complete freedom to choose how many of them should be activated at any time. It is with such understanding that AV-Test, one of the three testing labs involved, has stated that further investigation is ongoing, and will provide further updates on this topic.
 

hoang2007

Level 1
Thread author
Verified
Feb 22, 2015
23
Qihoo: "In any case, no alleged comment indicates that the level of protection from the product, is lower than the records achieved during the testing sessions."
3 labs' press release: "By contrast, as far as can be determined, all versions made generally available to users in Qihoo’s main market regions had the Bitdefender engine disabled and the QVM engine active. According to all test data this would provide a considerably lower level of protection and a higher likelihood of false positives."
Qihoo: "In addition, our product offers multiple engines, and regardless of the initial configuration, it is the user who has the complete freedom to choose how many of them should be activated at any time"
Fact: Users CANNOT disable QVM engine by any way.
Qihoo: "It is with such understanding that AV-Test, one of the three testing labs involved, has stated that further investigation is ongoing, and will provide further updates on this topic."
AV-Test: "We have now started to evaluate the possible manipulation of our performance testing. We have found strong evidence that another company, not Qihoo, is optimizing their product to do well in our performance test by excluding certain files and processes from checking. This is based on filenames and process names and can pose a security risk as well! We will check with AV-Comparatives and VB100 to verify our findings and will let you know as soon as we have the final data."
In conclusion: weak argument of 'Liar of the Year'. :D
 

akuigla

Level 1
Verified
Oct 29, 2013
40
China was the most developed country 4,000 years ago,2000 years ago,and will be again in few years.
Any product from China should be treated with,if not respect,at least with decent,reasonable attitude.
Bear in mind that everybody is innocent,until proven guilty.
 

FireShootSK

Level 17
Verified
Feb 17, 2015
824
China was the most developed country 4,000 years ago,200 thousand years ago,and will be again in few years.
Any product from China should be treated with,if not respect,at least with decent,reasonable attitude.
Bear in mind that everybody is innocent,until proven guilty.
Here is post from Qihoo FB

On 4/30/2015, Austria-based security testing lab AV-C accused Qihoo 360, along with Tencent and Baidu, of providing different versions of their security products for lab testing versus domestic distribution. Based on such accusation, AV-C revoked the rankings that Qihoo 360’s security products achieved in 2015. We believe the accusation and subsequent action by AV-C is without merit.

AV-C’s lab testing system is mostly based on the behaviors of European/Western Internet users, which may be significantly different from those of Chinese Internet users. For example, many popular software add-ons in China that are flagged as malware by the AV-C definition are in fact performing proper functions and not malicious. Therefore, Qihoo 360 and other domestic vendors’ security products in China treat such add-ons as legitimate and non-threatening. However, under AV-C testing logic, all domestic versions of security products in China are judged to be less effective since they fail to detect such “threats”. This is certainly not fair. To satisfy the security needs of both the domestic market and the need for foreign lab testing, all Chinese security product vendors make modifications to standard domestic versions for foreign lab testing to showcase the effectiveness of the basic protection capabilities of these products. In fact, a security product that strictly follows AV-C’s testing environment rule could be rendered useless in China due to the significantly different real-world environment.

We have been working with AV-C and other international testing labs for the last few years and we always keep those testing labs fully informed regarding the configuration and function of our testing products. That’s why we are a bit surprised by AV-C’s reaction to this particular incident. We are working with AV-C to solve this disagreement. As a market leader in Internet security in China, Qihoo 360 has been working diligently to provide effective protection for Chinese users in a very unique environment for the past 9 years. As a result of our efforts, China has become the safest Internet environment in terms of the malware infection ratio, according to a Microsoft study. We certainly intend to continue to do so with or without lab testing scores.

I like China, have very great food but this is terrible!
 

FreddyFreeloader

Level 32
Verified
Top Poster
Well-known
Jul 23, 2013
2,115
China was the most developed country 4,000 years ago,2000 years ago,and will be again in few years.
Any product from China should be treated with,if not respect,at least with decent,reasonable attitude.
Bear in mind that everybody is innocent,until proven guilty.
China will remain handcuffed by it's own paranoia, eg: The Great Firewall, and will remain a 2nd tier country until it opens the door of freedom of thought, speech, innovation, liberty.
 

aztony

Level 9
Verified
Oct 15, 2013
501
The allegation highlights that the default configuration of the product available for the public, differs from the configuration used by the labs for testing. This configuration was explicitly declared upon submission of the tests, and was thereafter confirmed by the test labs.
That's a pretty powerful, and damning statement. If true, Qihoo is stating that AV-C knew before, during, and after testing the product its characteristics and differences. Stay tuned, this sounds like a declaration of war Qihoo style.
 

hoang2007

Level 1
Thread author
Verified
Feb 22, 2015
23
Qihoo: " many popular software add-ons in China that are flagged as malware by the AV-C definition are in fact performing proper functions and not malicious ... (they) are flagged as malware by the AV-C definition ... To satisfy the security needs of both the domestic market and the need for foreign lab testing, all Chinese security product vendors make modifications to standard domestic versions for foreign lab testing to showcase the effectiveness of the basic protection capabilities of these products"
3 labs' press release: "On requesting an explanation from Qihoo 360 for their actions, the firm confirmed that some settings had been adjusted for testing, including enabling detection of types of files such as keygens and cracked software, and directing cloud lookups to servers located closer to the test labs."
Qihoo: "As a result of our efforts, China has become the safest Internet environment in terms of the malware infection ratio, according to a Microsoft study"
What study? Does that study conclude that China's Internet is safe thanks to Qihoo? China's Internet could be safe thanks to many other factors. Qihoo seems to boast itself too much!
 

akuigla

Level 1
Verified
Oct 29, 2013
40
Is Avast "safe"--- ?

http://www.howtogeek.com/199829/avast-antivirus-was-spying-on-you-with-adware-until-this-week/

Or McAfee,or Kaspersky,who was an intelligent officer in the Soviet Army?

http://www.telegraph.co.uk/technolo...rival-Kaspersky-is-much-more-interesting.html

How about Avg and its shady toolbars:

http://searchengineland.com/googles...s-is-bad-news-for-many-toolbar-players-147385

I am not saying it is true,but it seems to me, that Anti virus companies and Av testing establishments are well connected,finance each others,and make money in selling,testing,selling,testing...
 
  • Like
Reactions: Nirv5668 and aztony

Šovelén

Level 3
Verified
Mar 29, 2014
127
Yeah, yeah ... decisions, decisions ...
Me? I will continue to do my job and the computer i let continue defend 360TSE - quality AV, which for years did not disappoint me and offers free more than others for hard cash ... ;)
 
D

Deleted member 21043

China was the most developed country 4,000 years ago,2000 years ago,and will be again in few years.
Any product from China should be treated with,if not respect,at least with decent,reasonable attitude.
Bear in mind that everybody is innocent,until proven guilty.
This is completely irrelevant.

It doesn't matter what country it came from. If they cheated, they cheated. If they didn't, then they didn't. No matter if it came from China, UK, USA, Romania, Germany,.. It wouldn't give them rights to cheat!
 
Last edited by a moderator:

Šovelén

Level 3
Verified
Mar 29, 2014
127
Is Avast "safe"--- ?

http://www.howtogeek.com/199829/avast-antivirus-was-spying-on-you-with-adware-until-this-week/

Or McAfee,or Kaspersky,who was an intelligent officer in the Soviet Army?

http://www.telegraph.co.uk/technolo...rival-Kaspersky-is-much-more-interesting.html

How about Avg and its shady toolbars:

http://searchengineland.com/googles...s-is-bad-news-for-many-toolbar-players-147385

I am not saying it is true,but it seems to me, that Anti virus companies and Av testing establishments are well connected,finance each others,and make money in selling,testing,selling,testing...

Antivirus is a product like any other. For developers, it's a job and if it is good, it is normal to get paid for it. When it's good ... You also do not work for free right? ;)
 
D

Deleted member 21043

I am not saying it is true,but it seems to me, that Anti virus companies and Av testing establishments are well connected,finance each others,and make money in selling,testing,selling,testing...
But have they lied? I don't think they have (correct me if I am wrong). The actions they commit would have been covered by legal grounds (which is why people should read Privacy Policies, Terms and Conditions/Of use Agreements).
 
  • Like
Reactions: Kent

Šovelén

Level 3
Verified
Mar 29, 2014
127
But have they lied? I don't think they have (correct me if I am wrong). The actions they commit would have been covered by legal grounds (which is why people should read Privacy Policies, Terms and Conditions/Of use Agreements).

You know Friedrich Nietzsche ? ;)
 

jackuars

Level 27
Verified
Top Poster
Well-known
Jul 2, 2014
1,688
We knew all this started with the jealousy of it's competitors. They are trying to find one leak to blow it out of proportions and gain market share. The investigations are going on, Qihoo has stated their facts now, and users need not worry about this. I'm pretty much sure this is nothing but misunderstanding. Even if it isn't, this ain't going to be a big deal.
 

Nirv5668

Level 2
Verified
Mar 21, 2015
88
Qihoo: " many popular software add-ons in China that are flagged as malware by the AV-C definition are in fact performing proper functions and not malicious ... (they) are flagged as malware by the AV-C definition ... To satisfy the security needs of both the domestic market and the need for foreign lab testing, all Chinese security product vendors make modifications to standard domestic versions for foreign lab testing to showcase the effectiveness of the basic protection capabilities of these products"
3 labs' press release: "On requesting an explanation from Qihoo 360 for their actions, the firm confirmed that some settings had been adjusted for testing, including enabling detection of types of files such as keygens and cracked software, and directing cloud lookups to servers located closer to the test labs."

This explanation makes sense. If it is true that AV-C considers cracked software and keygens as malicious, I am really annoyed. I have never used cracked software or keygens, but this is not a security issue unless they are malicious (to the customer of the AV software- the user...not software, music, etc. companies). I don't want to link to it, but there is an interesting 'guide' on social engineering. Basically, its easy to spread malware when it can be sent and the recipient told "don't worry, the anti-virus alert is just because its cracked software." I always told people who believe this that they were wrong, AV software is not in the 'adult parental control' business, blah blah. I guess I was wrong : ) It doesn't completely explain why Qihoo's engine was disabled. I wonder if the same situation could apply in reverse- basically software is detected as malicious in China that would be counted as a false positive?
 

Nirv5668

Level 2
Verified
Mar 21, 2015
88
But have they lied? I don't think they have (correct me if I am wrong). The actions they commit would have been covered by legal grounds (which is why people should read Privacy Policies, Terms and Conditions/Of use Agreements).

You are probably right that they (testers and vendors) didn't do anything illegal. The question for me is if they should be held to a higher standard, given the importance and sensitivity of the information they are protecting. Kind of like doctors and drug companies- these relationships might need to be regulated. Or at least have more transparent disclosure. If I publish a scientific paper, I have to disclose if my cousin's dog's received a free biscuit from a company in any way related to my field (well, almost ;)). Maybe at least there could be mandatory guidelines on providing clear disclosures of anything that could potentially be perceived as a conflict.
 
  • Like
Reactions: jackuars
D

Deleted member 21043

Your opinion is completely irrelevant.
It shows arrogance of a young boy with shallow knowledge.
Good to know. ;)

Just because Qihoo is developed in China does not give it any leverage over products made in other countries or mean it should just be respected... However, if this is your opinion (that it should be respected because it's made in China) then that's fine, everyone has one. However, mine is the opposite way, I am afraid. Apologies if you think I have shallow knowledge. :D
 
Last edited by a moderator:
Status
Not open for further replies.

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top