Forums
New posts
Search forums
News
Security News
Technology News
Giveaways
Giveaways, Promotions and Contests
Discounts & Deals
Reviews
Users Reviews
Video Reviews
Support
Windows Malware Removal Help & Support
Inactive Support Threads
Mac Malware Removal Help & Support
Mobile Malware Removal Help & Support
Blog
Log in
Register
What's new
Search
Search titles only
By:
Search titles only
By:
Reply to thread
Menu
Install the app
Install
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Forums
Security
Security Statistics and Reports
Randomness in the AV Labs testing.
Message
<blockquote data-quote="Andy Ful" data-source="post: 916255" data-attributes="member: 32260"><p>You are irritated without the right reason.<img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite109" alt=":)" title="Smile :)" loading="lazy" data-shortname=":)" /><img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite130" alt="(y)" title="Thumbs up (y)" loading="lazy" data-shortname="(y)" /></p><p>All that I presented in this thread is already well known by any AV testing Lab. It is also written in plain text:</p><p>"<em>Our tests use much more test cases (samples) per product and month than any similar test performed by other testing labs. Because of the higher statistical significance this achieves, we consider all the products in each results cluster to be equally effective, assuming that they have a false-positives rate below the industry average</em>."</p><p><a href="https://www.av-comparatives.org/real-world-protection-test-methodology/" target="_blank">Real-World Protection Test Methodology - AV-Comparatives (av-comparatives.org)</a></p><p>As can be seen from the AV-Comparatives reports, the best AVs are usually in the first cluster (10 Avs or more), so they can be equally effective on malware in-the-wild (despite the differences in the particular test).</p><p></p><p>I think that you want the model presented in this thread to be more than it is. Like any model, it is based on some assumptions. These assumptions cannot be proved or rejected, because the real data are not known. They could be verified only when all data gathered by the AV vendors and AV testing Labs would be publicly known.</p><p></p><p>Anyway, even such a simple model is fully compatible with the AV-Comparatives Real-World rewards. In fact, the model is very similar to the cluster method used in the AV-Comparatives reports. The model and the cluster method simply say that when looking at the results of a single test, there is a big random factor. So, several AVs can in fact have different protection in the wild in the tested period, as compared with the scorings in the one particular test.</p><p></p><p>The model does not say that one cannot say anything interesting about AVs, especially when comparing the results of many reports.<img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite130" alt="(y)" title="Thumbs up (y)" loading="lazy" data-shortname="(y)" /></p></blockquote><p></p>
[QUOTE="Andy Ful, post: 916255, member: 32260"] You are irritated without the right reason.:)(y) All that I presented in this thread is already well known by any AV testing Lab. It is also written in plain text: "[I]Our tests use much more test cases (samples) per product and month than any similar test performed by other testing labs. Because of the higher statistical significance this achieves, we consider all the products in each results cluster to be equally effective, assuming that they have a false-positives rate below the industry average[/I]." [URL='https://www.av-comparatives.org/real-world-protection-test-methodology/']Real-World Protection Test Methodology - AV-Comparatives (av-comparatives.org)[/URL] As can be seen from the AV-Comparatives reports, the best AVs are usually in the first cluster (10 Avs or more), so they can be equally effective on malware in-the-wild (despite the differences in the particular test). I think that you want the model presented in this thread to be more than it is. Like any model, it is based on some assumptions. These assumptions cannot be proved or rejected, because the real data are not known. They could be verified only when all data gathered by the AV vendors and AV testing Labs would be publicly known. Anyway, even such a simple model is fully compatible with the AV-Comparatives Real-World rewards. In fact, the model is very similar to the cluster method used in the AV-Comparatives reports. The model and the cluster method simply say that when looking at the results of a single test, there is a big random factor. So, several AVs can in fact have different protection in the wild in the tested period, as compared with the scorings in the one particular test. The model does not say that one cannot say anything interesting about AVs, especially when comparing the results of many reports.(y) [/QUOTE]
Insert quotes…
Verification
Post reply
Top
