Forums
New posts
Search forums
News
Security News
Technology News
Giveaways
Giveaways, Promotions and Contests
Discounts & Deals
Reviews
Users Reviews
Video Reviews
Support
Windows Malware Removal Help & Support
Inactive Support Threads
Mac Malware Removal Help & Support
Mobile Malware Removal Help & Support
Blog
Log in
Register
What's new
Search
Search titles only
By:
Search titles only
By:
Reply to thread
Menu
Install the app
Install
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Forums
Battlefield
Software Comparison
Planned: Real-world Test of Trend Micro, ZoneAlarm, Eset and Webroot
Message
<blockquote data-quote="Andy Ful" data-source="post: 1098813" data-attributes="member: 32260"><p>[USER=99014]@Trident[/USER],</p><p></p><p>From the OP it follows that the test will not be Real-World. Of course, it does not mean that your test will not be interesting.</p><p>Making a Real-World test is hardly possible for a single tester.</p><ol> <li data-xf-list-type="ol">You have to keep the pule of samples realistic and representative, preserving the ratio of file types and malware types used in the wild in the testing period.<br /> This can be a serious challenge even for professional AV testing labs. For example, one shortcut as an initial vector can deliver different malware payloads hour after hour. Some AVs can detect that shortcut and protect against those payloads without knowing/detecting them. If you skip the shortcut in the test, the AV protection scorings will be biased. The same can be true for documents and scripts (including JavaScript code embedded/attached in emails).</li> <li data-xf-list-type="ol">You must test the malware immediately after it is seen in the wild to avoid dead samples. Dead samples are malware that infect people in the wild for a short time and disappear before the test is done (cannot infect anyone except the tested computer). With a delay of one day, most samples will be dead.</li> <li data-xf-list-type="ol">Dead samples can produce contradictory results because most of such samples can infect only one or two computers in the wild. For example, if the Bitdefender user was infected in the wild, Bitdefender can create the signature and detect the malware in the test. So for Bitdefender, that malware was initially FUD (Fully UnDetecteble), but not FUD at the test time. On the contrary, the same malware is still FUD at the test time for other AVs. To make this issue statistically insignificant one must use many random samples and fulfill the requirement from point 1. above.</li> </ol><p>All the above points (and some more) can make the test results different from the in-the-wild protection and produce unrealistic comparative results.</p></blockquote><p></p>
[QUOTE="Andy Ful, post: 1098813, member: 32260"] [USER=99014]@Trident[/USER], From the OP it follows that the test will not be Real-World. Of course, it does not mean that your test will not be interesting. Making a Real-World test is hardly possible for a single tester. [LIST=1] [*]You have to keep the pule of samples realistic and representative, preserving the ratio of file types and malware types used in the wild in the testing period. This can be a serious challenge even for professional AV testing labs. For example, one shortcut as an initial vector can deliver different malware payloads hour after hour. Some AVs can detect that shortcut and protect against those payloads without knowing/detecting them. If you skip the shortcut in the test, the AV protection scorings will be biased. The same can be true for documents and scripts (including JavaScript code embedded/attached in emails). [*]You must test the malware immediately after it is seen in the wild to avoid dead samples. Dead samples are malware that infect people in the wild for a short time and disappear before the test is done (cannot infect anyone except the tested computer). With a delay of one day, most samples will be dead. [*]Dead samples can produce contradictory results because most of such samples can infect only one or two computers in the wild. For example, if the Bitdefender user was infected in the wild, Bitdefender can create the signature and detect the malware in the test. So for Bitdefender, that malware was initially FUD (Fully UnDetecteble), but not FUD at the test time. On the contrary, the same malware is still FUD at the test time for other AVs. To make this issue statistically insignificant one must use many random samples and fulfill the requirement from point 1. above. [/LIST] All the above points (and some more) can make the test results different from the in-the-wild protection and produce unrealistic comparative results. [/QUOTE]
Insert quotes…
Verification
Post reply
Top
