Google Security Expert Criticizes Meaningless Antivirus Excellence Awards

[bunchuu]

Over the weekend, one of Google's top security researchers, Tavis Ormandy, published a blog post in which he criticized antivirus certification programs that award meaningless prizes to flawed security products.

His problem came from the fact that at this year's RSA security conference held at the start of March, Verizon's ICSA Labs awarded Comodo the 2016 Excellence in Information Security Testing Award.

The irony of this award wasn't lost on him, nor us, if we take into account that since last December, Mr. Ormandy has been unearthing security flaws in Comodo's Antivirus products on a regular basis.

Mr. Ormandy + Comodo = <3
The researcher first discovered that Comodo was forcibly installing an insecure browser that disabled SOP (Same-Origin Policy), a key security feature in Web browsers. He also learned that Comodo's scanning process didn’t enable ASLR protection, and then that the entire antivirus was using incorrect ACLs (Access Control Lists).

Later on, he also discovered that one of Comodo's tech support tools bundled by default with some of its security products was also installing an insecure VNC server with weak authentication, equally as bad as his first findings.

But the situation is even worse since all the above problems have been reported and fixed. Checking Mr. Ormandy's Twitter feed today, we still see unresolved issues. The most recent of them (see tweet below), allows an attacker to exfiltrate keystrokes just by scanning a file.

Working on an unusual exploit for Comodo Antivirus, just *scanning* a file can exfiltrate keystrokes. #wtf pic.twitter.com/NKmPGh2DMW — Tavis Ormandy (@taviso) March 10, 2016
It is to no surprise that Mr. Ormandy had a problem with Verizon giving Comodo an award for excellence in information security, since if you follow his research that doesn't seem to be a feature of Comodo's antivirus to begin with.

Some antivirus certification tests are just hilarious
But besides Comodo, Mr. Ormandy also took to heart the criteria which Verizon used to certify the high standards of information security which Comodo had to pass.

Since Verizon published its methodology according to which the awards were given out, Mr. Ormandy was quick to point out that they were extremely simplistic.

This reporter also took a look at all the criteria, and in our assessment, most antivirus products would have passed since the certification requirements merely described basic antivirus functions, half of which were related to UI functions.

Some of the certification "criteria" included the likes of: "Enable and disable the Detection of Malware" (which is a basic start/stop button for the scanning process), "Retrieve and apply the latest Engine and Signatures over the Internet" (the antivirus must be able to update itself), "On-Demand Detection" (the antivirus must start a scan when you press a button, or a new file is detected), and "Report no false positives" (well, duh!).

Most antivirus products are a collection of deprecated codebases
But Mr. Ormandy's criticism was not only directed at Verizon and Comodo, and he said that antivirus products, in general, are insecure. "All of the major security vendors are using ancient codebases with no awareness of modern security practices, it’s still hacking like it’s 1999," the researcher said.

And he's right in his assessment. Before cutting Comodo's reputation to bits, Mr. Ormandy also discovered security issues with other security products from companies such as Avast, Malwarebytes, Trend Micro, AVG, FireEye, Kaspersky, and ESET.

He did all of his research with no access to source code, with point-and-click security tools, and basic techniques which every security researcher learns.

His point is that the vendors of these security products do very little testing on their own codebases, and leave trivial security holes that are extremely easy to detect for any security expert, and probably hackers as well.

The Google researcher says that both antivirus vendors and certification programs would do everyone a favor if they'd follow basic security testing procedures like the ones put forward by Microsoft's SDL (Microsoft Security Development Lifecycle).

"There’s no need to reinvent the wheel here. [...] Many of these [security tests] don’t require any skill and can be automated, but would actually be useful," Mr. Ormandy explains. "Award bonus points in some ranking for using sandboxing, and maybe we'll see the first vendor actually implement that."

source
source

this information make us put more doubt to AV lab testing and AV development.

So, Do you still believe AV certification?

 

[Kiwimike]

I'm actually REALLY happy someone said this.

Because I think Antiviruses suck too, I think they are insecure as hell. After working for AVG for a few years I have gained a bunch of knowledge of not only antivirus software but also that antiviruses kind of stink.
Often if you are a security expert you are better off not using an antivirus. Antivirus can expose vulnerabilities on your system and expose it to more threats. This is not because they want to do that it's because software is exploitable.

And so it makes sense that the more software you INSTALL (installing itself into the filesystem, Registry and running in the background) can increase vulnerabilities. This is why I don't use antivirus, I prefer trusting myself and not using antivirus software. You also learn a lot more that way because you if you have to find out whether a random application is malware without using any scanners it can increase your malware-analysis skills significantly.

And that is another reason why I don't use antivirus.

But to the point, Are AV certifications bogus?
I don't beileve they are entirely bogus. I just think it's a really stupid game, because it's almost like they praise those who are the good guys. I think more credit should be given to the smaller guys who make software which is actually good. You're better off patching your system, using a good firewall (Windows Firewall) and keeping Windows Defender.

Windows Defender has some significant improvements, if you want to use a companion scanner but seriously, Windows Defender protects a lot better some a lot of antiviruses currently when you go in depth. It integrates with Secure Boot for example. so right after the OS is verified Windows Defender boots, this is a great feature and not only that but having an antivirus integrated into the OS makes it much more stable, secure and allows it to integrate with actual Microsoft Techologies.

 

[Morvotron]

I've lost all my hope on AV testers (say AV-COMPARATIVES, AV-TEST and every one of them). Firstly, it was obvious and easy to see how tests were guided by the company that offered the bigger reward to the testing organization. As a matter of fact, it's imperative to highlight that those antivirus vendors who get in discussions with this kind of certificators are often magically disqualified. (gonna search the web and give you later the link to one of this articles). To example, it's hilarious to see how AV-COMPARATIVES has not tested Symantec products in years. Norton is beating the market lately.

What Google's expert says is totally true. Most antivirus keep downgrading their quality, and as times passes we see more and more malicious sample get missed. I remember back when antivirus used to lock and stop lots of malware out there. Now, if i'm not mistaken, i've read security softwares are blocking less than 10% actual malware. Still, one has to use one to at least be protected.

If we talk about Comodo, it's something i've always said. Comodo Internet Security on itself is a very low-quality level product. Firewall is basic, still it offers some nice control over the registry. And support is simply useless. Everyone who has every contacted them can tell you their only aim is to sell you products, even when you've bought their main one. I remember the first time i used Comodo Support tool built on the firewall, this tech guy used remote access to touch two or three files, and tell me he had to fix it, so i had to buy the product to get assisted. He even opened the browser, and displayed the Comodo Official Website. Navigated through it and clicked on BUY. And he kept writing: "now here put your credit card number, here the security code". He wouldn't even leave the control acess software to let me do it.

To conclude, i think the best tests are done by ourselves. Those ones are the smartest and most clever, since softwares do not behave the same on every system.

 

[Online_Sword]

I am curious about what security config would Google's expert use if he was required to use Windows? (Well I know they use Mac at work...I am just curious )

 

[bunchuu]

I'm actually REALLY happy someone said this.

Because I think Antiviruses suck too, I think they are insecure as hell. After working for AVG for a few years I have gained a bunch of knowledge of not only antivirus software but also that antiviruses kind of stink.
Often if you are a security expert you are better off not using an antivirus. Antivirus can expose vulnerabilities on your system and expose it to more threats. This is not because they want to do that it's because software is exploitable.

And so it makes sense that the more software you INSTALL (installing itself into the filesystem, Registry and running in the background) can increase vulnerabilities. This is why I don't use antivirus, I prefer trusting myself and not using antivirus software. You also learn a lot more that way because you if you have to find out whether a random application is malware without using any scanners it can increase your malware-analysis skills significantly.

And that is another reason why I don't use antivirus.

But to the point, Are AV certifications bogus?
I don't beileve they are entirely bogus. I just think it's a really stupid game, because it's almost like they praise those who are the good guys. I think more credit should be given to the smaller guys who make software which is actually good. You're better off patching your system, using a good firewall (Windows Firewall) and keeping Windows Defender.

Windows Defender has some significant improvements, if you want to use a companion scanner but seriously, Windows Defender protects a lot better some a lot of antiviruses currently when you go in depth. It integrates with Secure Boot for example. so right after the OS is verified Windows Defender boots, this is a great feature and not only that but having an antivirus integrated into the OS makes it much more stable, secure and allows it to integrate with actual Microsoft Techologies.

For me, I still rely on AV since I don't have adequate skill and expertise in computer security. I think AV dev shall improve their software and I agree that WD improved significantly after W10 launched.

 

[_CyberGhosT_]

So more or less AV certification is now being run by politicians in the format of special interest donations lol
Seriously though I get it and agree, this has not been a secret for quite some time now, The fact that this is an award offered by Verizon is laughable, I will get excited and cause a stir when notable AV labs start pushing Comodo as "cream of the crop". PeAcE
Nice share bunchuu, Thank You.

 

[hjlbx]

This is an important fact quoted from above:

"His point is that the vendors of these security products do very little testing on their own codebases, and leave trivial security holes that are extremely easy to detect for any security expert, and probably hackers as well."

Anyone who is familiar with security softs knows this implicitly after being around for a while. The low-level, guinea-pig user, beta testing model that the security soft industry relies so heavily upon just doesn't cut it.

With more and more reports about all security soft suites, it is obvious that industry standards leave something to be desired. The quality of security softs - with all their "advanced features" - is really suspect.

However, a lot of the security soft vendors state that most of these type of findings - while, indeed, a vulnerability - are difficult for the typical malware publisher to exploit. The word typical is important - since the typical malware publisher isn't very sophisticated according to the industry.

This stuff should be certainly thought-provoking, if not confusing, to the user who just wants to protect their system.

The bottom line is this - if a user configures their system and introduces unknown files to the system very little, makes informed decisions and uses good judgment online - then those safe behaviors - will go a very long way in mitigating most of the security risks. Add a few security and related softs based upon actual usage needs, and the typical safe user will never see an infection.

Anyone who doubts this position, just ask @Huracan. He used only Windows Defender and Windows Firewall for over 6 years, and never got infected.