App Review Windows Defender vs Ransomware 2024 (TPSC)

[blackice]

You can find out by running benchmarking software and doing things like running apps or browsing folders with lots of files in external drives. In my case, Kaspersky is usually the lightest and Defender the heaviest, with the rest in between. YMMV.

Methodology, etc., for one test:

Performance Test October 2023

AV-Comparatives released the Performance Test report October 2023 for consumer security products under Microsoft Windows 10.

www.av-comparatives.org

I think this is extremely dependent on the system. I have a 5800x with 32GB of ram (so pretty high end still) and I haven't seen a difference between any AV's in benchmarks (3D mark, Cinebench, Futuremark), with the exception of AVG/AVAST which always inexplicably has a lower CPU score for 3D Mark benchmarks. Defender is the same as ESET on my system. So I put little stock into AV-Comparatives performance ratings. They can be misleading.

 

[SeriousHoax]

"gang bang" approach is a valid test method. There are malware downloaders out there called "monster installers" which rapidly download and install a bunch of malware simultaneously.

Did you just create an account to comment this? I prbably know who you are, and you have defended Leo's testing method before using the same monster installer logic. Nothing against you but these "monster installer" are extremely rare to the point that they are irrelevant. If Leo did a mixture of gangbang vs one by one testing, then I could maybe ignore that but no, he only tests one way. He even has to whitelist his malware launching script in certain AV products (eg: Bitdefender) to make his test work. So, it's not a real-world scenario or an acceptable testing method.

 

[oldschool]

The gangbang approach is a faulty method as we have discussed before.

This is one real problem with his test, and he insists on using it because it makes his video tests more "professional" looking.

 

[Templarware]

In what way?

A difference in daily usage, opening the browser, websites, file explorer... And back when I was having some trouble with Hogwarts Legacy, I suspect it was also hurting gaming performance.It's faster with Kaspersky Free, for example. Hardened defender is even slower than default.

 

[Xeno1234]

I think this is extremely dependent on the system. I have a 5800x with 32GB of ram (so pretty high end still) and I haven't seen a difference between any AV's in benchmarks (3D mark, Cinebench, Futuremark), with the exception of AVG/AVAST which always inexplicably has a lower CPU score for 3D Mark benchmarks. Defender is the same as ESET on my system. So I put little stock into AV-Comparatives performance ratings. They can be misleading.

Kaspersky has same issue with 3d mark CPU benchmark - decreased score by around 10%

 

[Shadowra]

I've said it many times before, but bombarding the antivirus with the "gangband" effect DOES NOT HELP!
By the time the resident has processed the various threats, it can let some through (and I've already seen Leo's examples with the Expiro virus in a Kaspersky test).

Always scan the pack and then launch with a 30s interval (which is what I do).

 

[upnorth]

Spoiler: TPSC Headquarter :p

 

[ErzCrz]

Spoiler: TPSC Headq[SPOILER="OCC

uarter "]View attachment 281981

[/SPOILER]

Hehe. I have a OCC t-shirt I bought years ago. Doesn't fit anymore but they were great

Anyway, I take his approach to the tests with the automation with a pinch of salt. I echo what @Shadowra said about allowing time between samples.

 

[Andy Ful]

Tamper protection has become stronger but from what I see, that doesn't cover exclusions. So, exclusions can still be added on Windows 11 systems. If such malware can bypass Defender's pre-execution static analysis, then it can't stop them from adding exclusions to it. Even some legit programs do this to avoid performance impact.

Administrator rights are required, so it is not so easy on SUA. Anyway, I think that it should be an option in Tamper Protection to disable adding exclusions. In this way, the exclusions would be protected and the user could still add some exclusions after disabling Tamper Protection temporarily.

 

[oldschool]

I think that it should be an option in Tamper Protection to disable adding exclusions. In this way, the exclusions would be protected and the user could still add some exclusions after disabling Tamper Protection temporarily.

That would be a good suggestion for the Feedback Hub, especially coming from you.

 

[oldschool]

@Andy Ful I went ahead and submitted your feature request. All MT members are encouraged to upvote this on the Feedback Hub, FWIW.

 

[simmerskool]

You can find out by running benchmarking software and doing things like running apps or browsing folders with lots of files in external drives. In my case, Kaspersky is usually the lightest and Defender the heaviest, with the rest in between. YMMV.

Methodology, etc., for one test:

Performance Test October 2023

AV-Comparatives released the Performance Test report October 2023 for consumer security products under Microsoft Windows 10.

www.av-comparatives.org

the numbers don't lie, but I also do not see or notice any slowdown running MD, at least lately. But maybe I'm not doing the things you mention where the slowdown is seen.

 

[Andy Ful]

"gang bang" approach is a valid test method. There are malware downloaders out there called "monster installers" which rapidly download and install a bunch of malware simultaneously.

The "gang bang" approach can be a valid test method if the author mentions that the video is a special stress-kind test. The results of such a test can hardly impact overall protection and it should be mentioned in the test.
Of course, the stress test with ransomware does not make any sense. It should be done on general malware samples or even better on "monster installer" samples (if such exists in considerable numbers).

The author (and other authors) did not do any of the above, so I think that considering the video as a protection test is unjustified. If someone would like to see the video as a valid AV protection test, then it could be considered a hoax. I prefer to see it as a good presentation.

 

[monkeylove]

I think this is extremely dependent on the system. I have a 5800x with 32GB of ram (so pretty high end still) and I haven't seen a difference between any AV's in benchmarks (3D mark, Cinebench, Futuremark), with the exception of AVG/AVAST which always inexplicably has a lower CPU score for 3D Mark benchmarks. Defender is the same as ESET on my system. So I put little stock into AV-Comparatives performance ratings. They can be misleading.

What scores did you get in contrast to those of AV-Comparatives, and did you publish the results?

 

[monkeylove]

That one miss wouldn't have happened if he manually ran each sample one by one. So, it's nothing to do with increasing Defender's protection. The gangbang approach is a faulty method as we have discussed before. In a perfect scenario of course, Defender should have stopped that ransomware since they already have signatures for it. So, it's a strange behavior to miss that because products like Avast, Bitdefender, ESET, Kaspersky, etc. don't miss detecting samples for which they already have local signatures. But since this testing method is not a real-world scenario, we can somewhat ignore it. Defender have other dangerous issues like malware adding exclusions to it.

I guess the antidote to a "gangbang approach" is to test the AV in default mode using the malware to which it failed. From there, tweak and argue that users are foolish for not messing around with the settings or argue that statistically one will not be hit by that malware so they'll do fine.

 

[blackice]

What scores did you get in contrast to those of AV-Comparatives, and did you publish the results?

I actually sent them to AVG. It’s been a while since I’ve compared many. The last time I did several was probably late 2022. Last year I found no difference between F-Secure and Defender.

 

[monkeylove]

I actually sent them to AVG. It’s been a while since I’ve compared many. The last time I did several was probably late 2022. Last year I found no difference between F-Secure and Defender.

What scores did you get in contrast to those of AV-Comparatives, and did you publish them? I don't know how we'll be able to get them from AVG.

 

[blackice]

What scores did you get in contrast to those of AV-Comparatives, and did you publish them? I don't know how we'll be able to get them from AVG.

Honestly I probably deleted the screenshots because they had served their purpose of my own personal information gathering. My comment was anecdotal. But as a long time user of Microsoft Defender I have not seen the impact they show. I know certain developers do see severe IO issue. That’s not something I ever do. Opening apps, startup, and most daily tasks are pretty much the same with any solution I’ve tried. When Avast/AVG was rated as very low impact by AV-Comparatives I saw an approximate 8% worse performance in CPU intensive tasks like handbrake, or benchmarking like Cinebench. The only security software I have experienced that was demonstrably lighter on CPU load was ESET, and that was about 2%. That’s with a fairly modern 8 core 16 thread processor. Laptops or less powerful desktops might see much higher impacts. Again I was giving my experience from memory. I’m not trying to convince you of anything, just noting that I have found the AV-Comparatives performance metrics mostly useless for my use case.

 

[Andy Ful]

I guess the antidote to a "gangbang approach" is to test the AV in default mode using the malware to which it failed.

I am afraid that It would be an invalid method. Defender (and some other AVs) can miss a sample for several reasons (not necessarily due to "gang bang"), but the detection can be sometimes corrected after a few minutes. So when you test the samples missed in the first test, some of them will be correctly detected even when "gang-bang" was not the reason for initial failure.
A better method would be to test initially all samples in default mode, and then use the "gang-bang" test on correctly detected samples.

 

[Andy Ful]

I’m not trying to convince you of anything, just noting that I have found the AV-Comparatives performance metrics mostly useless for my use case.

Defender's low performance in the last AV-Comparatives test mostly follows from archiving/unarchiving. So, most users will not see the difference. Some users can feel the difference, but the reasons are unclear and this can happen also for other popular AVs. On one of my computers, Avast has slightly better performance than Defender.

Edit.
In older performance tests, Defender had lower performance related to file copying and installing applications. Also in these cases, many users will not see slowdowns in daily work.
For average users, the important performance factors are:
Launching Applications, Downloading Files, and Browsing Websites.
Those factors are related to 99% of daily actions.