Status
Not open for further replies.
D

Deleted member 178

Zurchiboy said:
I think they should test it at a level they think will give the user maximum protection without FPs.
The advantage of setting in Max protection for a test, is that you can also see which one has the highest FPs.

Protection tests are supposed to be just tests, not simulate real world conditions for basic users, anyway no users will encounter hundreds of samples in a day/week/month so test labs testing FPs rates/usability using lot of samples are wrong already.

If they want to test usability, they have to put a basic user on a pre-configured computer with a solution set as default and let him use it as he want for a month, sometimes give him a list of url/exe (some safe, some malicious) and let him react alone.

You want realistic test, you must do as a real world environment.
 

Zurchiboy

New Member
That is true. it will also show the most sensitive products. I also agree that an excellent usability test would be to put a novice user in the pre-configured computer. I don't know how reliable he automated tests done by AV-comparatives are. usually the most popular products do well in the tests.
 
D

Deleted member 178

For example, everybody says Windows Defender/MSE is bad or has low detection, but in real "real world test" as i described earlier i'm sure it will finish in the top with the less FPs.
 

Zurchiboy

New Member
Microsoft do have very few FPs. they also have good signatures as well, not the best but good for a company that isn't 100% antivirus.
 

Littlebits

Retired Staff
Testing sites try to show an complete picture that will apply to the majority users which are about 99% novice by testing all aspects of each solution.

Testing only protection features will not show the complete picture it can be very deceptive to novice users. We all know that solutions with higher detection rates usually have higher false positives as well and are not as user-friendly. But novice user do not know this.

User-friendliness is the most important factor, if a user can't figure it out then it is useless to them even if the solution has the highest detection rate. It just takes a few false positives to render a system non-operational for a user who can't tell the difference.

If you Google you can find many examples of how false positives have caused serious problems for users more so then what most malware infection can cause.

I'm sure that most testing sites don't want to do tests that only apply to less then 2% of the total users. They have to do testing that applies to the majority. I know that might be hard for some advanced users to understand but that is just the way it is.

When you break down the aspects, protection features are not one of the most important factors to determine what is best for any user. I know that might sound crazy to some of you but it is the truth if you want to apply it to the majority.

Thanks.:D
 

jamescv7

Level 61
Trusted
Verified
The security survey could adapt the way they test from all security products so assuming those test varies from standards and request.
 
D

Deleted member 178

Any protection test should not include the users ability, just the solution performance against threats.

Now if they want see how a solution performs when used by basic users, the test should be different and not called "detection, protection,etc..."

it is why all those tests are biased to me, not saying the financial transactions that may get involved...
 

Littlebits

Retired Staff
Umbra Corp. said:
Any protection test should not include the users ability, just the solution performance against threats.

Now if they want see how a solution performs when used by basic users, the test should be different and not called "detection, protection,etc..."

it is why all those tests are biased to me, not saying the financial transactions that may get involved...
In that case UAC would get 100% on all testing sites and Microsoft would be #1.

Of coarse many users don't know how to UAC properly and will just click approve which will fail them. See the importance of including all aspects in AV test?

This would also apply to all HIPS, sandboxing and virtualization as well but what does that mean to most users who don't understand how to use them?

I do agree that even though AV testing test against most aspects their results really don't help a user to decide what is best for them and some testing sites are biased and influenced by money.

I have customers that use various AV products (MSE, Avast, AVG, Avira, Norton, McAfee, BitDefender, etc.) and what I have found that it doesn't really apply to any certain product which one gives the user the best protection, the main protection applies to the user actions.

User who don't use precautions will always get infections no matter what product they use. Many users who do take precautions and MSE never get infections. That is why I believe just about any AV can protect a user who uses precautions, for those who don't use precautions there is no hope for them no matter what product they use. That is why I believe AV tests are irrelevant. Nothing will give these users 100% protection. I have had customers bring in their systems with AV's installed and updated with the highest protection scores on AV testing sites loaded full of malware.

Thanks.:D
 
Status
Not open for further replies.