- Mar 29, 2018
- 7,106
Is the improved performance of Microsoft Defender a myth? Should we necessarily be using a 3rd party AV?
- Thread starter geek166
- Start date
Please provide comments and solutions that are helpful to the author of this topic.
- Dec 23, 2014
- 8,129
Of course, you have the right to have your personal opinion about any AV, but there is no need to seek fanboys among people who disagree with you. Also, your preference of Kaspersky over Defender free with default settings is OK.
The methodology similar to adopted in video tests was discussed and criticized many years ago, by people from the AV industry (like Eset, Kaspersky, etc.). This discussion caused the testing Labs to adopt AMTSO methodology. The video demonstration tests are as far from this methodology as Earth from the stars. Such methodology cannot be a source for the comparison of many popular AVs.
For example, if a video test uses 1000 samples, then one of the AMTSO requirements is using 1000 VM snapshots - each snapshot for any single sample. I did not see any video-test based on many samples, that could fulfill even 20% of needed AMTSO requirements.
Last edited:
- Dec 23, 2014
- 8,129
Are all the video tests useless?
No, they are not. I found some of them useful. I like the video tests that can show a particular weakness in AV protection. There were many such tests on the MT forum. A few PC Channel tests can be among them. For example:
malwaretips.com
There is a sufficient difference in the results of the online and offline detection/protection, that shows Defender's dependence on the cloud backend. Of course, this test is fully compatible with the results of professional tests made by AV-Comparatives (member of AMTSO).
This test can also show how big have to be differences between results to take a test seriously, even if it does not fulfill the AMTSO methodology.
The cons of most video tests are the author's comments and conclusions. They can suggest something that does not truly follow from the test results.
No, they are not. I found some of them useful. I like the video tests that can show a particular weakness in AV protection. There were many such tests on the MT forum. A few PC Channel tests can be among them. For example:
App Review - Windows 11 vs Ransomware [TPSC]
malwaretips.com
There is a sufficient difference in the results of the online and offline detection/protection, that shows Defender's dependence on the cloud backend. Of course, this test is fully compatible with the results of professional tests made by AV-Comparatives (member of AMTSO).
This test can also show how big have to be differences between results to take a test seriously, even if it does not fulfill the AMTSO methodology.
The cons of most video tests are the author's comments and conclusions. They can suggest something that does not truly follow from the test results.
Last edited:
Good thing I didn't say disable, even though a good chunk of malware disable Windows Defender, way more than the 5% you claim, as for Windows Firewall is not needed to be disabled.
- Dec 23, 2014
- 8,129
The discussions about Defender free is usually a mess because people put together the home environment and business environment. This follows from the fact that Defender free is usually compared to commercial AVs. In fact, such a comparison should be done with Microsoft 365 Defender or Microsoft Defender for Endpoint.
www.infusedinnovations.com
There is convincing evidence that some malware tried to change Defender's settings and Windows Firewall settings to get persistence in the business network. These samples were used in the targeted attacks often as payloads after compromising the business network by another malware. I did not see any evidence that such samples were used in widespread attacks. If one knows it then please let me know.
In the business environment, using Defender free on default settings and Windows Firewall on default settings are not good solutions.
This is not in contradiction with that for most users in the home environment the difference between Defender free + Edge + Windows Firewall and Microsoft Defender for Endpoint can be hardly important.
Microsoft Defender For Endpoint Standalone is Now Available! (Formerly MD ATP)
Note: Microsoft Defender Advanced Threat Protection (MD ATP) Is Now Defender For Endpoint Update: As of August 2021, MDE P1 is also available bundled in Micros
www.infusedinnovations.com
There is convincing evidence that some malware tried to change Defender's settings and Windows Firewall settings to get persistence in the business network. These samples were used in the targeted attacks often as payloads after compromising the business network by another malware. I did not see any evidence that such samples were used in widespread attacks. If one knows it then please let me know.
In the business environment, using Defender free on default settings and Windows Firewall on default settings are not good solutions.
This is not in contradiction with that for most users in the home environment the difference between Defender free + Edge + Windows Firewall and Microsoft Defender for Endpoint can be hardly important.
Last edited:
- Dec 23, 2014
- 8,129
Closer to the topic.
The test results over the time for Defender free according to AV-Test:
People believed in AV-Test results for Defender free in the years 2013-2017. It is strange that they have a problem with believing the results in the years 2018-2021.
The same for Kaspersky Internet Security:
The test results over the time for Defender free according to AV-Test:
People believed in AV-Test results for Defender free in the years 2013-2017. It is strange that they have a problem with believing the results in the years 2018-2021.
The same for Kaspersky Internet Security:
Last edited:
- May 29, 2021
- 383
In all the years since win95, we have all lived with microsoft's problems over and over again. When we thought that now it works, then the blue screen came along with other errors. This is still happening, so I think I have to be very gullible to believe that WD should be the exception when it comes to errors in a Microsoft product. Since Andy posted the summary of the tests for AV-Test, I will add f-secure's tests regarding protection from 2010-2021.
- Dec 23, 2014
- 8,129
You probably read my older post which is even more convincing, because it uses the results of all 3 Labs (AV-Test, AV-Comparatives, and SE Labs):
https://malwaretips.com/threads/the-best-home-av-protection-2019-2020.106485/#post-927440
It would be hard to differ between Kaspersky and F-Secure.
- Dec 23, 2014
- 8,129
People who take video tests seriously (as reliable protection tests) can have a problem understanding that in the home environment (and Real World scenario) there could exist AV with 100% protection, that would detect only 50% malware/phishing URLs and only 50% drive-by malware. Of course, the opposite is not true.
That is why people invented AMTSO requirements for testing.
That is why people invented AMTSO requirements for testing.
Last edited:
- Mar 29, 2018
- 7,106
- Jan 28, 2018
- 2,463
What I wonder about the Defender. Originally, I feel that software like these boosters should have been prepared by Microsoft on its own.
You wasting your time with personal attacks and poor arguments, the fact this is not the first time and I don't have time nor patience for kids /blocked
The test labs are safe environments to advertise Anti-Virus software, it wouldn't surprise that the major Anti-Virus companies would want tests to follow their specifin guidelines, contrary to random videos online showing the raw materials.People who take video tests seriously (as reliable protection tests) can have a problem understanding that in the home environment (and Real World scenario) there could exist AV with 100% protection, that would detect only 50% malware/phishing URLs and only 50% drive-by malware. Of course, the opposite is not true.
That is why people invented AMTSO requirements for testing.
The AMTSO requirements weren't made by people, but by the AV companies themselves. They should be taken as a grain of salt, and not real world benchmarks.
In the real world WD with default settings is still weak in comparison to the competition, and I wouldn't trust it to defend any family member machine.
This is not even an Enterprise argument that was brough for no reason, WD with ATP is a far different beast.
- Aug 21, 2020
- 475
The reality is Windows killed most malware with their newer more frequent upgrade cycle. Malware devs cannot catch up with how fast Windows devs are closing the holes. By the end of this decade, AVs will be irrelevant.
- Jan 27, 2018
- 1,261
Is a fanboi the opposite of a Microsoft basher?
- May 29, 2018
- 2,606
- May 3, 2015
- 1,535
- Dec 7, 2021
- 402
I always use a 3rd party AV & probably always will on my Pc(s) & any I fix for others - I don't really have a good reason, some is habit, some I remember how truly dreadful Defender used to be, now changed - However a lot of things I do I in life I really do not have a compelling reason for apart from that's my choice which maybe I feel is as good a reason as any
- Dec 23, 2014
- 8,129
Any proof, except for conspiracy theory?
It is probable that sometimes the test can be biased by sponsors. But, we have democracy. If one AV vendor could pay to bias the results, others can do (and will do) the same. So, over a longer time, you still have unbiased (on average) results. Furthermore, such things usually cannot be kept hidden for a long time. So far, all Labs mentioned by me have a very good reputation.
Furthermore, there is no proof that in some cases the video tests are biased by sponsoring. We know for sure that the interpretation of these tests made by authors can be biased too (for many reasons).
Yes, the AMTSO requirements were made by AV companies and not by video makers. Also, crash car tests were made by engineers, hired by car companies, and were not made by amateurs. Thank God for that. Are AMTSO requirements perfect? Probably no, but they are sensible, on the contrary to the testing methodology (or rather mythology) of video tests.
Are you moving in the circle?
The Defender is not good ---> so it cannot get good results (despite the AV Lab tests) ---> AV Lab tests are biased ----> The video tests are not biased (why???) ---> Some video tests show that Defender is not good ---> The Defender is not good.
You even reject any test result that is good for Defender (like some of Leo's tests).
That is what I said many times, but I have no idea why you put this comment here.
Last edited:
- Dec 23, 2014
- 8,129
To the MT readers.
Please be patient. My posts are directed not only to @Local Host. Many people do not realize how complex is reliable AV testing. It is not sufficient to throw many malware samples and count how many of them were detected. There are many other important things to do that are skipped in the video tests. The final one is a statistical analysis of how reliable are the results. This is a similar situation to the election forecast. Any company that makes such a forecast has to calculate a statistical error, which usually is about a few percent. There is always such an error because we still do not know the votes of most people.
So, if one has 1000 malware samples and the testing methodology causes 5% statistical error, then the AV that is protected against all samples can have the same protection in the wild as the AV which failed on 50 samples.
If one takes into consideration this factor, then magically almost all video tests are compatible with professional tests (video test wonder).
The professional AV Labs do such analysis. It is visible in the final recommendations. So, if AV-Test gives the 6 points to 10 AVs, it means that the testing methodology cannot say which one is better (even if they missed a different number of malware samples).
Please be patient. My posts are directed not only to @Local Host. Many people do not realize how complex is reliable AV testing. It is not sufficient to throw many malware samples and count how many of them were detected. There are many other important things to do that are skipped in the video tests. The final one is a statistical analysis of how reliable are the results. This is a similar situation to the election forecast. Any company that makes such a forecast has to calculate a statistical error, which usually is about a few percent. There is always such an error because we still do not know the votes of most people.
So, if one has 1000 malware samples and the testing methodology causes 5% statistical error, then the AV that is protected against all samples can have the same protection in the wild as the AV which failed on 50 samples.
If one takes into consideration this factor, then magically almost all video tests are compatible with professional tests (video test wonder).
The professional AV Labs do such analysis. It is visible in the final recommendations. So, if AV-Test gives the 6 points to 10 AVs, it means that the testing methodology cannot say which one is better (even if they missed a different number of malware samples).
Last edited:
Is extremely simple, all this tests are useless, is extremely easy to write lines of codes to pass testing, that would make the program useless in the real world.
Pretty much how all those AD-Blocker tests work, they simply block the testing page, same applies to malware testing in this case, they using simple malware that in no way replicates how Home Users get infected.
Nowadays most people just get their browser hijacked, and there's nothing WD will do prevent that (Kasperksy on the other hand can detect and rollback the changes).
Cars are constantly compared, but they don't compare to one another, either way, even car testing is optimized for specific scenarions, and there's a good chunk of cars out there that will kill you in a crash.
I never claimed anywhere video testing is the holy grail, nor that they aren't biased either, if anything I claimed exactly the opposite, and I not wasting time arguing on that.
Simple, always using the home environment excuse for WD short comings and saying is enough, fact remains third party AVs are superior to WD still,
Similar threads
