Reply to thread

Message

<blockquote data-quote="danb" data-source="post: 636582" data-attributes="member: 62850">Cool, thank you for the 100 samples!I noticed that there are A LOT of duplicates in the 100 samples you sent me.&nbsp; Typically, I think it is important to remove duplicates before testing, simply because if there are 10 samples that are duplicates, in a test of only 100 samples, it is going to skew the results heavily one way or the other.&nbsp; For example, if the software you are testing misses that sample, it skews the results so that the reported efficacy is much lower than the true efficacy.&nbsp; Conversely, if the software you are testing nails the sample, it skews the results so that the reported efficacy is much higher than the true efficacy.&nbsp; If you are testing 1,000-3,000 samples, it does not matter quite as much, but either way, it is best to remove the dups.&nbsp; Does that make sense?I have a small utility I wrote that removes the duplicates, I will paste a link to it once I find it.&nbsp; It is not super advanced, but it does a pretty good job of removing the duplicates.Also, when testing VS, it really does require an internet connection... for the blacklist scan and for the VoodooAi scan.&nbsp; Our ML/Ai models are in the Azure ML platform... we do not have local models.&nbsp; We might one day, but since VS does not require updates that often (in theory <img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite110" alt=";)" title="Wink&nbsp; &nbsp; ;)" loading="lazy" data-shortname=";)" />), it is nice to have our ML models in the cloud, so we can update them without updating the client software.If you test VS without an internet connection, it should block every single one of them, and tell you that an internet connection is not found.&nbsp; If you experience something different from that, please let me know <img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" class="smilie smilie--sprite smilie--sprite110" alt=";)" title="Wink&nbsp; &nbsp; ;)" loading="lazy" data-shortname=";)" />.&nbsp; Thank you!</blockquote>

[QUOTE="danb, post: 636582, member: 62850"] Cool, thank you for the 100 samples! I noticed that there are A LOT of duplicates in the 100 samples you sent me. Typically, I think it is important to remove duplicates before testing, simply because if there are 10 samples that are duplicates, in a test of only 100 samples, it is going to skew the results heavily one way or the other. For example, if the software you are testing misses that sample, it skews the results so that the reported efficacy is much lower than the true efficacy. Conversely, if the software you are testing nails the sample, it skews the results so that the reported efficacy is much higher than the true efficacy. If you are testing 1,000-3,000 samples, it does not matter quite as much, but either way, it is best to remove the dups. Does that make sense? I have a small utility I wrote that removes the duplicates, I will paste a link to it once I find it. It is not super advanced, but it does a pretty good job of removing the duplicates. Also, when testing VS, it really does require an internet connection... for the blacklist scan and for the VoodooAi scan. Our ML/Ai models are in the Azure ML platform... we do not have local models. We might one day, but since VS does not require updates that often (in theory ;)), it is nice to have our ML models in the cloud, so we can update them without updating the client software. If you test VS without an internet connection, it should block every single one of them, and tell you that an internet connection is not found. If you experience something different from that, please let me know ;). Thank you! [/QUOTE]

Verification