Reply to thread

Message

<blockquote data-quote="Andy Ful" data-source="post: 1022681" data-attributes="member: 32260">Not exactly. The fluctuations are always big if the number of tested samples in one test is very very small compared to the number of total samples. Even significant changes in the population cannot change it (until the number of tested samples is relatively small).You can see this on the charts of AV-Comparatives, AV-Test, and SE Labs tests (over the years 2019-2022 and probably before this period). Furthermore the term &quot;more or less stable&quot; can depend on the unknown-yet statistical model, which can be not much sensitive to the changes in the population.Starting from this point, we can increase the number of tests and observe if the fluctuations decrease. The changes in the population are now unimportant because we take the cumulative number of missed samples over a much longer period (many tests).The changing population can have some impact when we compare the results of two periods, like 2019-2020 with 2021-2022. But no one knows how big it can be.Did you notice that this calculator is insensitive to the changes in the population? If you use a population size 100 times bigger (6000000), then you get the sample size 385 ??? It can be an example that significant changes in population can be not important for some statistical models.From the testing methodology (for example AV-Comparatives), we know that one test (about 300 samples per month) cannot differentiate between AVs contained in the same award group (usually 2/3 of tested products). So, several tests are required.This calculator uses a particular statistical model that is clearly incompatible with the data related to AV tests.</blockquote>

[QUOTE="Andy Ful, post: 1022681, member: 32260"] Not exactly. The fluctuations are always big if the number of tested samples in one test is very very small compared to the number of total samples. Even significant changes in the population cannot change it (until the number of tested samples is relatively small). You can see this on the charts of AV-Comparatives, AV-Test, and SE Labs tests (over the years 2019-2022 and probably before this period). Furthermore the term "more or less stable" can depend on the unknown-yet statistical model, which can be not much sensitive to the changes in the population. Starting from this point, we can increase the number of tests and observe if the fluctuations decrease. The changes in the population are now unimportant because we take the cumulative number of missed samples over a much longer period (many tests). The changing population can have some impact when we compare the results of two periods, like 2019-2020 with 2021-2022. But no one knows how big it can be. Did you notice that this calculator is insensitive to the changes in the population? If you use a population size 100 times bigger (6000000), then you get the sample size 385 ??? It can be an example that significant changes in population can be not important for some statistical models. From the testing methodology (for example AV-Comparatives), we know that one test (about 300 samples per month) cannot differentiate between AVs contained in the same award group (usually 2/3 of tested products). So, several tests are required. This calculator uses a particular statistical model that is clearly incompatible with the data related to AV tests. [/QUOTE]

Verification