D
Deleted member 178
Thread author
(This is my personal opinion, i am not in the business so i may be wrong; feel free to think different.)
Lately lot of users (especially on this forum) strongly rely on AV test labs (like AV-Comparative, AV-Test, Matousec, etc...) to choose their security solution, that should be a good behavior but the problem is that they mislead you. why?
Basically, Testing organizations use malware samples found in the wild, install the product in default setting in its own system, then froze the signature update by disabling the internet connection, then launch the malwares against the product to test its proactivity & detection following a distinct procedure.
what you -users- must know:
A- non-paranoid view
1- Product Features:
Each products use different ways to protect you:
- Real-time scanner Aka the "engine" using signature and heuristics, unfortunately for test labs , each vendors has its own delay before adding the latest malwares signatures, so when they froze the updates, a vendor may not add yet some sigs so it will failed the test. For this point it is quite normal, but what you should know is that some vendors uses only the recent known most propagated malwares, if a lab decide to use old samples (retired from the database because no more considered as a threat) or those active in some regions only, the product may fail.
- Proactive features aka Behavior Blocker/HIPS/Web Filter/Cloud, etc...
again, some products don't use any of them and are still compared with the ones that have it...major failure, you can't compare uncomparable things. products should be tested separately depending their features.
- Settings, each products is set out of the box with what we use to call "default settings" , most labs test them with default settings; it is again a failure since some products are set almost at maximum by default when others don't. To be meaningful test-labs should set every tested product with minimal or maximal setting.
Testing Labs need to be able to configure multiple products to provide comparable protection in each of them to offer legitimate results. Since no products offer the same/similar levels of protection, comparing them is quite useless.
2- Malwares samples
tests-labs use hundreds of malwares in a row, real world users will never encounter that sums of malwares in their whole life, and no products is supposed to be targeted with that sums of malwares.
B- Paranoid View
Of course, i have no proof of what i will write below (even if we know some of this facts), but it does not mean it is not possible
1- Business
Security is a big business that generate lot of money, either for cyber-criminals than for security vendors, since Labs' results will influence potential customers for the best rated products, how -us, simple users- could we trust those labs, even if they said they are "independent" (words are just words). Money is money and corruption is everywhere...even some labs are known to give pre-results to some vendors so they can update their products in exchange for some "contributions" then the official result are publicly released if it satisfies the vendor...
Final Note:
I will just give this advice, don't rely "only" on Testing Organizations, take them as "one piece of information"; try a product for 2-3 weeks then decide if you like it enough to buy a license.
Thanks
Lately lot of users (especially on this forum) strongly rely on AV test labs (like AV-Comparative, AV-Test, Matousec, etc...) to choose their security solution, that should be a good behavior but the problem is that they mislead you. why?
Basically, Testing organizations use malware samples found in the wild, install the product in default setting in its own system, then froze the signature update by disabling the internet connection, then launch the malwares against the product to test its proactivity & detection following a distinct procedure.
what you -users- must know:
A- non-paranoid view
1- Product Features:
Each products use different ways to protect you:
- Real-time scanner Aka the "engine" using signature and heuristics, unfortunately for test labs , each vendors has its own delay before adding the latest malwares signatures, so when they froze the updates, a vendor may not add yet some sigs so it will failed the test. For this point it is quite normal, but what you should know is that some vendors uses only the recent known most propagated malwares, if a lab decide to use old samples (retired from the database because no more considered as a threat) or those active in some regions only, the product may fail.
- Proactive features aka Behavior Blocker/HIPS/Web Filter/Cloud, etc...
again, some products don't use any of them and are still compared with the ones that have it...major failure, you can't compare uncomparable things. products should be tested separately depending their features.
- Settings, each products is set out of the box with what we use to call "default settings" , most labs test them with default settings; it is again a failure since some products are set almost at maximum by default when others don't. To be meaningful test-labs should set every tested product with minimal or maximal setting.
Testing Labs need to be able to configure multiple products to provide comparable protection in each of them to offer legitimate results. Since no products offer the same/similar levels of protection, comparing them is quite useless.
2- Malwares samples
tests-labs use hundreds of malwares in a row, real world users will never encounter that sums of malwares in their whole life, and no products is supposed to be targeted with that sums of malwares.
B- Paranoid View
Of course, i have no proof of what i will write below (even if we know some of this facts), but it does not mean it is not possible
1- Business
Security is a big business that generate lot of money, either for cyber-criminals than for security vendors, since Labs' results will influence potential customers for the best rated products, how -us, simple users- could we trust those labs, even if they said they are "independent" (words are just words). Money is money and corruption is everywhere...even some labs are known to give pre-results to some vendors so they can update their products in exchange for some "contributions" then the official result are publicly released if it satisfies the vendor...
Final Note:
I will just give this advice, don't rely "only" on Testing Organizations, take them as "one piece of information"; try a product for 2-3 weeks then decide if you like it enough to buy a license.
Thanks