Detection Rate of Security Products?

[Littlebits]

Since I have been reading through the forums for awhile now and also other security based forums, I have repetitively seen users posts about detection rate. Like "product A" has a better detection rate of "product B".

I would like to know where you all get this information from?

Hopefully your not using AV tests sites, because they only test samples. There is no way any testing site could be able to give you an accurate detection rate of any product. They may give you some kind of incite on how products may do on certain malware samples but that isn't enough information to base your decision on whether to use a certain product over another product. For example, on your system you may never encounter any of the malware used in the testing samples but may encounter other malware that wasn't even tested. Even malware tests with a large samples would only include a small faction of know and unknown malware in the wild.

Then there are video reviews which even use less malware samples than what most testing sites use. Of coarse video reviews are entertaining, they may show you how a product will protect against certain malware and how to use a product, but once again you may never encounter any of the malware used in the video review but could encounter other malware not in a video review.

So the bottom line is nobody knows what any products detection rate is.
It is how the product that your using protects you is what matters.

If a user can be protected by just using a basic free AV and Windows Firewall then they have no need adding anything else. Advanced users should require less protection than novice users because they should know what "not to do". However it appears just about on every security forum, I see the opposite. Advanced users think they have to drive a tank to be protected. Like a professional swimmer wearing several life jackets because he might drown.

Just by using knowledge in your online habits will increase your detection rate, don't depend on just your software doing all the work for you.

Thanks.

 

[win7holic]

don't depend on just your software doing all the work for you.

I agree with you. absolutely, Common Sense is needed for now and forever.

for me, I've tried and using Avast free and Norton AV on my new laptop start from 10 march 2011 until now, never get infections also I ever test AV on this machine for myself. and do anything with this laptop, it (Avast or NAV) was great for me. Because, my common sense and Prevention or detection rate works perfectly.

how about you all?

 

[moonshine]

I have tried a lot of products through the years and I'm yet to get any infection, Thanks to my knowledge and common sense.

 

[MetalShaun]

When people say detection rate they mean signatures. Obviously Prevention is what is most important. But signatures are still the easiest and user friendly way of detecting malware. If there was a product that could detect 100% of malware with signatures, none of us would be fannying around with Hips etc.

Hopefully your not using AV tests sites, because they only test samples. There is no way any testing site could be able to give you an accurate detection rate of any product.

So the bottom line is nobody knows what any products detection rate is.

I disagree, its about collecting evidence. By collating all the information from different testing sites, and user experience, anecdotes etc. You can get a fairly accurate idea of how the said AV will perform in the wild. For example, No one would listen if someone came to these forums arguing that eScan has better detection than Avira Antivir, because the overwhelming evidence suggest it doesn't.

Cheers
Shaun

 

[HeffeD]

When people say detection rate they mean signatures. Obviously Prevention is what is most important. But signatures are still the easiest and user friendly way of detecting malware. If there was a product that could detect 100% of malware with signatures, none of us would be fannying around with Hips etc.

I think detection rate means detection rate...

A heuristic detection is still a detection and has nothing to do with signatures, yet it will be counted towards the detection rate.

When people do AV tests to determine detection rates, (which are indeed subjective, as has already been mentioned...) do they turn off heuristics? I've never seen them do this. They're usually tested with default settings, and 99% of the products out there have heuristics on by default because it's long been known that signatures are never going to be able to keep up with malware releases.

However, by their very nature, (think of them as a behavior blocker for your AV) heuristics are prone to false positives. Which is why you don't see many pure heuristic AV products, and why AV's generally don't recommend cranking heuristics up to their highest setting.

 

[Littlebits]

When people say detection rate they mean signatures. Obviously Prevention is what is most important. But signatures are still the easiest and user friendly way of detecting malware. If there was a product that could detect 100% of malware with signatures, none of us would be fannying around with Hips etc.

Hopefully your not using AV tests sites, because they only test samples. There is no way any testing site could be able to give you an accurate detection rate of any product.

So the bottom line is nobody knows what any products detection rate is.

I disagree, its about collecting evidence. By collating all the information from different testing sites, and user experience, anecdotes etc. You can get a fairly accurate idea of how the said AV will perform in the wild. For example, No one would listen if someone came to these forums arguing that eScan has better detection than Avira Antivir, because the overwhelming evidence suggest it doesn't.

Cheers
Shaun

Of coarse it can give you some kind of idea but to apply that idea to yourself is another thing.
Some Avira users may get more infections than eScan users because of their online habits therefore that info is irrelevant to them.

I think detection rate means detection rate...

A heuristic detection is still a detection and has nothing to do with signatures, yet it will be counted towards the detection rate.

I agree, detection rate should include all means used to detect malware.

heuristics are prone to false positives.

Also agree, you rarely get false positives from signatures but some AV heuristics are a false positive paradise.

Thanks.

 

[MetalShaun]

When people say detection rate they mean signatures. Obviously Prevention is what is most important. But signatures are still the easiest and user friendly way of detecting malware. If there was a product that could detect 100% of malware with signatures, none of us would be fannying around with Hips etc.

I think detection rate means detection rate...

A heuristic detection is still a detection and has nothing to do with signatures, yet it will be counted towards the detection rate.

When people do AV tests to determine detection rates, (which are indeed subjective, as has already been mentioned...) do they turn off heuristics? I've never seen them do this. They're usually tested with default settings, and 99% of the products out there have heuristics on by default because it's long been known that signatures are never going to be able to keep up with malware releases.

However, by their very nature, (think of them as a behavior blocker for your AV) heuristics are prone to false positives. Which is why you don't see many pure heuristic AV products, and why AV's generally don't recommend cranking heuristics up to their highest setting.

Heuristics can use signatures depends what methods are used but I fully agree with your point.
Maybe I should have said pattern matching (or something else) because I meant heuristics as well. Anything that can positive flag a file as Malicious. Without the user having to make a decision. That's what I think the majority of people class as detection.

Some Avira users may get more infections than eScan users because of their online habits

That is true, but they are still better protected using Avira then if they used eScan.

 

[Jack]

Detection rate means nothing these days, as their is so much malware thrown at us that there is a very high chance that a reputable antivirus engine might miss a threat . As far as I've notice from my tests almost 80% of the zero day threats can be detected by the antivirus engine (and here I can I;m including the heuristic engine) however the other 20% will need to be stop by another layer of protection.
As we've seen a lot of vendors have implemented and are continually developing new alternative ways to stop zero day malware. For what is worth I do think that Symantec with their Insight Tehnology and COMODO with their Auto-Sandbox are on the right path on the war against malware.

 

[ZeroDay]

In this day and age no matter how safe your browsing habbibts are and no matter how confident one feels, hips, sandboxing and other 'extra' security measures are an absolute must. We've all gone long periods whithout infection,but,for me the key is to never get to confident because lets face facts it takes one zeroday malware to cause you a whole load of trouble.

If anyone believs their knowlege and favourite av are enough they are sadly mistaken, that of course is just my opinion.

 

[Littlebits]

In this day and age no matter how safe your browsing habbibts are and no matter how confident one feels, hips, sandboxing and other 'extra' security measures are an absolute must. We've all gone long periods whithout infection,but,for me the key is to never get to confident because lets face facts it takes one zeroday malware to cause you a whole load of trouble.

If anyone believs their knowlege and favourite av are enough they are sadly mistaken, that of course is just my opinion.

You all need to know the likelihood of actually getting infected with a zero-day malware. It's not very likely if you have good habits, you are more likely to win the lottery. One thing that most people neglect to consider, almost every zero-day infections has to be manually downloaded and manually installed, they just don't jump off a web page and attack. Therefore your own knowledge should protect you better than your security solution against zero-day malware.

Zero-day infections are more common to novice users who are tricked into downloading a file from a malicious site. Most zero-day infections are just rogue security products, ransomware, other rogue products that don't really do much harm to your system more annoying than anything.

I mess the days when zero-day malware were actually something to be concerned about.
Zero-day malware back about 5 years ago and longer included System Killers, Worms, Trojans, Destructive Viruses, etc.

You can read reports from various security vendors about how more people get infected now than what they did from the past. But what the reports fail to mention is how most malware now days is non-destructive rogueware, in the past malware was much more dangerous. The reports try to make it sound like malware infections are getting worse, when they are actually getting much better. Maybe as an attempt to sale their products or cause paranoid users to panic.

I'm older than most of you, I have been dealing with malware infections since 1998. I can say without any doubts that malware now days is much easier to deal with, causing much less damage and is much easier to remove. The only thing that has got worse is the number of fake infected websites which distribute rogueware and how most users can easily be tricked into downloading and installing these rogueware. Most security products just can't keep up with these rogueware, that is why Malwarebytes is the main removal tool for most computer techs. MBAM is one of the only security products that can detect and remove rogueware easily.

I would like to see a security product from now days take on a System Killer or a destructive malware like the viruses from the early 2000's. I know who would win.

The so-called detection rate can not be applied to all users the same way, it all depends on user habits. I have used various security products throughout the years some which according to the testing sites have a low detection rate and still failed to get infected that is without any sanboxing or HIPS featured products. Of coarse users would pay more attention to what they download and install if they had to deal with the older destructive malware, just one infection and you didn't have to worry about removing it because your system and hard drive was trashed.

Thanks.

 

[Deleted member 178]

In this day and age no matter how safe your browsing habbibts are and no matter how confident one feels, hips, sandboxing and other 'extra' security measures are an absolute must.

i totally agree with Jack and Zeroday, how can your knowledge and habits can protect you from something nodody knows or see...

 

[LochNess]

In this day and age no matter how safe your browsing habbibts are and no matter how confident one feels, hips, sandboxing and other 'extra' security measures are an absolute must.

i totally agree with Jack and Zeroday, how can your knowledge and habits can protect you from something nodody knows or see...

Agree

 

[Littlebits]

In this day and age no matter how safe your browsing habbibts are and no matter how confident one feels, hips, sandboxing and other 'extra' security measures are an absolute must.

i totally agree with Jack and Zeroday, how can your knowledge and habits can protect you from something nodody knows or see...

What is really sad, I have some novice customers with good habits that only use Microsoft Security Essentials and Windows Firewall and they haven't had any infections at all in the last 5 or 6 years now. I have other customers that use Avast Free, Avira Free or AVG Free with Windows Firewall that never get any infections either. They just use one simple rule, don't download files or install .exe's from unknown sources.

My nephew now 17 years old has had his laptop for 4 years now. He just uses Microsoft Security Essentials and Windows Firewall for security. Uses Opera browser with some security add-ons and has yet to get one single infection. He browses all over the web, into gamming and cheap codes, I'm sure he has visited bad sites many times, but he never downloads files and doesn't install anything without asking his dad or myself. He is about as novice as they come, doesn't ever understand the concept of tab browsing.

Good habits will protect you much better than any security software. The problem is most people wants to or is tricked into adventuring into bad territory and that is what gets them into trouble.

I know there are a lot of paranoid users who think they have to use HIPS or sandboxing applications to be safe, but that is absolutely not true. Some of my customers, my family members, friends and myself are living proof that you can be protected just by using basic AV, Windows Firewall, secure browser with security add-ons and good habits.

I'm not the smartest in the malware category but neither is some of my customers or family members and friends, but if we can be malware free without using HIPS or sandboxing applications then others should be able to do the same especially advanced users.

Thanks.

 

[Ramblin]

They just use one simple rule, don't download files or install .exe's from unknown sources.

Might be a simple rule but following it, keeps me clean. I think of installs as my Achilles heel because I run just about everything sandboxed. Basically, only installs and updates are done out of the sandbox.

It does not matter whether the user uses an AV or not or uses Sandboxie instead, being extra careful with downloads and installs will help keep users clean no matter whats the strategy used.

Bo

 

[illumination]

I have to agree with Jack and the few others that believe "layered" security is the way to go now days, for novice and advanced users. Why take the chance of being infected, especially if it means infecting others on your network and or social sites you are visiting. It is true that a big percent of infections come from downloading, but there are other means of getting infected. Lets say the product you are running has a 99.4% detection rate, well this is good to a point, that small percentage while appearing to be nothing, can become extremely big when one considers the mass amount of new malware released into the wild on a daily basis, and is not yet been discovered or added to the data bases yet. this is where sandboxing, hips, and other extra security comes into play. This is my personal opinion, and will be the way i always approach security, better safe then sorry..

 

[Littlebits]

I have to agree with Jack and the few others that believe "layered" security is the way to go now days, for novice and advanced users. Why take the chance of being infected, especially if it means infecting others on your network and or social sites you are visiting. It is true that a big percent of infections come from downloading, but there are other means of getting infected. Lets say the product you are running has a 99.4% detection rate, well this is good to a point, that small percentage while appearing to be nothing, can become extremely big when one considers the mass amount of new malware released into the wild on a daily basis, and is not yet been discovered or added to the data bases yet. this is where sandboxing, hips, and other extra security comes into play. This is my personal opinion, and will be the way i always approach security, better safe then sorry..

Could you please explain the "other means of getting infected" besides downloading files or malicious scripts on webpages that can be blocked with secure browser with add-ons?

Here in the last 5 years or so, I haven't seen any other ways malware can infect a system.
Something must be downloaded period, the days of malware automatically downloading files is now gone with more secure browsers, even Internet Explorer doesn't allow files to automatically download. Files have to be manually downloaded by the user.

I do agree with "layered" security which includes using a respectful AV, Windows Firewall, Secure Browser with add-ons and the most important is user habits with downloading files and installing programs.

Some advanced users make themselves targets because they like to explore the unknown more often than basic users, because yes they might benefit from using advanced HIPS or sandboxing.

Advanced users like myself try to avoid exploring the unknown and if I do want to test a unknown file, I will open it in a sandbox application like Sandboxie or Avast sandbox. If the file is suspected to be malicious, then I will open it in VirtualBox or Returnil System Safe Free.

For everyday real-time protection, I simply don't need HIPS or sandboxing because I never get infections.

Thanks.

 

[HeffeD]

Could you please explain the "other means of getting infected" besides downloading files or malicious scripts on webpages that can be blocked with secure browser with add-ons?

USB sticks, any variety of flash RAM drive, CD's, DVD's, offline storage, network shares, etc...

 

[Hungry Man]

Keep this in mind when you think that common sense will protect you:
80% of sites that spread malware are hacked legitimate sites.

That means that you may very well be visiting a completely legitimate site that got hit by some automated SQL injection and now it's distributing malware. This happens of the time, usually to no-name nothing sites but sometimes to big ones. MySQL.com was hit once - that's a popular site.

What happens then?

Well your "common sense" goes out the window.

"Common sense" often will dictate "Don't visit a sketchy site. Only allow what you have to."

Let's say I visit a site that often uses Java (not Javascript, let's be clear!) so I allow applets in my NoScript or whatever. Suddenly that site gets hit by an exploit. I visit it thinking la-di-da it's a legit site. Suddenly some 0day Java exploit (and those happen a LOT) downloads a payload to my computer!

Gasp!

This has happened. Recently a porn site was hacked among a few others. All delivered Java exploits using a very very new exploit at the time - no patch had been made available. People had used the sites before, they were always legitimate. This time they weren't. How could they have known?

Now, with something like NoScript or sandboxed iFrames you can mitigate plenty of these sites that are hacked but it's nto bullet proof by any means whatsoever. Chrome users and Firefox users were all effected by this exploit. Some lucky Firefox users hadn't allowed much of the site via NoScript but most just sorta "blanket bombed" the site with full rights until their videos start to play. The nice thing about hacking a porn site is you're dealing with someone who's impatient and not interested in security popups! lol

Almost every user whose computer I helped clean up/ who reported the infection to me was running MSE/ Panda/ ESET Nod32. All followed fairly smart/ safe computing practice - they had programs up to date.

You can't rely on common sense because it's the first thing hackers exploit. There are only two facets to security, technology and policy - common sense, market share, whatever, they don't play into the game.


EDIT: Furthermore, your point about browsers now being more secure is spot on. But plugins still aren't. Firefox users are subject to all Flash and Java exploits. Chrome users are subject to many Java exploits as well. So where do the attacks go? Java and Flash.

Reader exploits are actually a lot less common now with the advent of browser-PDF viewers and Adobe protected mode but they still happen.

 

[Littlebits]

Could you please explain the "other means of getting infected" besides downloading files or malicious scripts on webpages that can be blocked with secure browser with add-ons?

USB sticks, any variety of flash RAM drive, CD's, DVD's, offline storage, network shares, etc...

Simple solution, disable Autoruns and Network Shares on all of your drives. (As far as I known Microsoft already has disabled Autoruns and Network Shares by default with Windows Updates).

Most respectful AV will protect against portable devices and CD/DVD drives, of coarse you would have to manually run the malicious file in order to get infected then Windows would display file is not digitally signed notification. Some simple checking before running would be advised.

For website hacking, flash, PDF files and Java vulnerabilities. It is very rare to get infected by these means, I've been using the web since it was online and never got an infection from these means.
If I did, I'm sure I could do a one-time removal without any problems.

I know it is possible, but it is also possible that when you leave your home you might get ran over by a vehicle. So do you prepare to get ran over each time you leave your home or is that something that you don't think about?

So say you do get infected by these means, since advanced users can effectively configure complicated HIPS, sandboxing, etc. how hard would it be for these advanced users to clean the infection?

Thanks.

 

[Hungry Man]

Disabling autoruns works... unless the malware makes use of a 0day, as Stuxnet and a few others have done, which has allowed them to traverse through loads of computers.

For website hacking, flash, PDF files and Java vulnerabilities. It is very rare to get infected by these means, I've been using the web since it was online and never got an infection from these means.
If I did, I'm sure I could do a one-time removal without any problems.

Well I'm glad that you've never run into it. I've seen it literally hundreds of times. Within one month I saw a single Java exploit used to infect easily over 100 computers.

I assure you that this isn't some rare occurrence. The difference between being hit by a car and being hit by a drive-by is that there aren't groups of highly skilled drivers driving through my front door trying to kill me - it's purely chance. Users are targeted by hackers, they spray as many sites as possible and if they can get something popular they will.

If the exploit is trapped in a sandbox it's a matter of deleting the contents. If you use something like EMET the exploit may not even work. This is harder with Java because of how JIT exploitation works but because so many of the exploits rely on the VM being vulnerable you can actually mitigate many of them with EMET.