AVLab.pl May 2021 - Advanced In The Wild Malware Test

[ForgottenSeer 72227]

I don't want to drag this thread down much further, but I just want to echo what @Andy Ful said.
I myself am a MD user....im quite happy with it and it does its job quite well. It has excellent protection both at defaults and configured and I personally don't experience any of the performance issues that others have had.

In saying this...while I prefer to use MD and the built in security of Windows, I don't hate 3rd party AVs...they're just not my personal preference atm. That doesn't mean the're terrible, its just my use case.

In saying this, as good as MD has become and continues to improve, there can be an argument made to say that most people will be fine with MD. Which isn't wrong, it really is at the point where most people would be fine with using MD. However...that would be in a perfect world where all computers are the same and everyone's use case is exactly the same. We live in the real world where all computers are not the same and everyone's use cases/needs and wants are all different. So it is good to have 3rd party AVs because it gives us choice. It also forces compition to make better products overall.

MD doesn't work for everyone and that's ok...it doesn't mean it's bad. As good as it is, the UI and access to settings leaves a little to be desired. Furthermore, it's performance impact seems to be very case dependent...affecting some users over others. Also 3rd parties may have certain features built in (ie: VPN) that someone likes to use.

All in all there's no such thing as a perfect product...both in detection and overall usage. It always best to just try each one for yourself and use the one that meets your needs. If it's MD great, if it's a 3rd party that's great too.

 

[Andy Ful]

When we're looking at AV-Comparatives' benchmarks for performance over the past a few years, the MD is always the last.

Saying so is somewhat misguiding. The case is more complicated. In fact, when taking into account also the benchmarks of AV-Test and MRG Effitas, we can see that Defender benchmarks are much better than for average AV.

Let's look into the last Performance Test (April 2021) made by AV-Comparatives. There are only a few AVs (Eset, K7, Microsoft, Total Defense) that scored the maximum in the activities most important to average users: "Launching Applications", "Downloading Files", "Browsing Websites". These activities take over 99% of the time.
It is true that Microsoft is the slowest AV (in this test) for such activities as "File copying", "Archiving / Unarchiving", and "Installing Applications".
So the truth is that for most users Defender has got 99% of top performance and maybe 1% bottom performance. Of course, for some users this 1% can be annoying as @SeriousHoax already noticed.

When we look into the results of the last Performance Test (April 2021) made by AV-Test, we can see even better scorings (Defender scored top result 6.0):

Defender and Eset have got the top performance also in MRG Effitas test (Q1 2021):

In this test, Microsoft Defender has got the top performance also for other activities (not measured by AV-Comparatives and AV-Test) like: Bootup time, Size on disk, AV update.

Anyway, the AV performance can be different on different machines. Furthermore, the users usually have a personal view on which "performance factor" is most annoying.

 

[Andy Ful]

AV-Comparatives (as well as others') performance tests have never been reliably replicated outside of the lab.

That is true for all similar tests.
Anyway, the results of AV-Comparatives Performance tests are close to what I can see on my machines (I use SSD and 8GB RAM). In 99% of time it is a demon of speed ( "Launching Applications", "Downloading Files", "Browsing Websites") and in 1% of time I can feel that it is slower than top AVs ("File copying", "Archiving / Unarchiving", and "Installing Applications"). I can also understand why some people can feel Defender annoying. In some cases, the slow behavior of Defender is hard to improve without changing hardware/software. This topic was discussed several times on MT for several years.

 

[Local Host]

That is true for all similar tests.
Anyway, the results of AV-Comparatives Performance tests are close to what I can see on my machines (I use SSD and 8GB RAM). In 99% of time it is a demon of speed ( "Launching Applications", "Downloading Files", "Browsing Websites") and in 1% of time I can feel that it is slower than top AVs ("File copying", "Archiving / Unarchiving", and "Installing Applications"). I can also understand why some people can feel Defender annoying. In some cases, the slow behavior of Defender is hard to improve without changing hardware/software. This topic was discussed several times on MT for several years.

You agree and disagree with him at the same time, the tests are accurate, looking at the final result instead of the testing mythology that makes people believe the tests are not accurate.

I actually take the time to read the fine print, and can say the tests are as accurate as my real world experience.

 

[Andy Ful]

You agree and disagree with him at the same time,

Yes, I agree with one thing and disagree with another.

the tests are accurate, looking at the final result instead of the testing mythology that makes people believe the tests are not accurate.

I actually take the time to read the fine print, and can say the tests are as accurate as my real world experience.

In my opinion, all tests mentioned by me are accurate. I am not surprised that the results are similar to your experience. They are close to my experience too. But, this does not mean that the results cannot be different for other people.

From the AV-Comparatives test report:
"Please note: We want to make clear that the results in this report are intended only to give an indication of the impact on system performance (mainly by the real-time/on-access components) of the consumer security products in these specific tests. Users are encouraged to try out the software on their own PC’s and see how it performs on their own systems."

 

[ForgottenSeer 85179]

I wonder if limiting Defender CPU power to 10% will solve the "performance" problem.

 

[artek]

The biggest issue any user is going to experience is a software incompatibility caused by the AV, which is far less likely using Windows Defender over third party products. If there are no issues they're going to be fairly similar in terms of performance.

 

[roger_m]

MD has virtually zero impact in my experience. And there certainly is background resource usage by Antimalware processes, but it rarely every causes a system slow down. Only when certain newly released processes are launched does the latest iterations of Antimalware cause a delay.

Sure, that's your experience. But many others have a very different experience. I can give examples of antiviruses that I've found to exceptionally light and some users have found to be quite heavy. On the other hand, there are antiviruses which people often find to be light, which I find to be very heavy on my computer. This is to be expected, due to how much antivirus performance varies from one system to the next. I've even witnessed very noticeable differences in performance on two computers with similar specs running the same antivirus.

Even you said the following.

The tests are accurate for only the systems used to perform the testing. The test results cannot be extrapolated to all other Windows systems. Doing so is absurd.

Obviously, this applies to MD, just like any other antivirus. If it works well for you, that's great, but it doesn't mean it will work the same for everyone else. But, because you insist that it's a waste of time using other antiviruses, and I have no idea why you do so, you fail to acknowledge the performance issues that a lot of people face. It's quite strange really, since no one here is saying MD is a terrible antivirus, or not to use it. I've made it quite clear that it's fine to use MD or a third party antivirus.

Do traces and you will see that the CPU usage is tied to disk use. So the type of drive is relevant.

Very few people have problems with Defender.

This is not the case. CPU and disk usage are two separate issues. I've seen antiviruses cause very noticeable slowdowns due to heavy disk usage, while the CPU use remains very low.

As for problems with MD, you only need to read posts about it here, to see that plenty of people have issues with it.

 

[Digmor Crusher]

Sikalinga makes sense in that when they do the performance testing it only applies to the one computer they are doing it on, hardware, software etc is different for every computer and to try to extrapolate it to millions of computers is a fools game. The same as when people say an AV is light or heavy on their system, what applies to their computer means nothing to anyone else's computer. And I get a feeling that when most people say "light or heavy" they have no clue, its mostly a preconceived idea they had in the first place. I know for me to determine if anything is light or heavy on my computer that it would have to be a considerable slowdown to even notice it.

 

[roger_m]

The same as when people say an AV is light or heavy on their system, what applies to their computer means nothing to anyone else's computer. And I get a feeling that when most people say "light or heavy" they have no clue, its mostly a preconceived idea they had in the first place. I know for me to determine if anything is light or heavy on my computer that it would have to be a considerable slowdown to even notice it.

I can't speak for others, but I do notice a huge amount of difference in the performance impact between some antiviruses. That hasn't always been the case, but it has been for a number of years now. With some antiviruses, my computer runs just about as well as if I'm running no antivirus at all. With a few, my computer runs significantly slower and with plenty of others, it's somewhere in between. In addition to that, I find that some are very light a lot of the time, but on occasion cause slowdowns due to high CPU use with not apparent reason. There are very few antivirus which I find are extremely light pretty much all the time.

Maybe if I had a much faster system, I would not see that much difference. But since I don't, I need to choose my antivirus carefully, so I don't get unacceptable performance drops.

 

[Behold Eck]

Even just knowing that MD uses more resources than some 3rd party AV`s makes me install something lighter as a matter of course. Ever since Win7`s Windows Defender days it`s one of the first tasks for me to do after a fresh install.

On an old net pad with windows 7 starter,1 gb of ram, I installed K7 Total Security and there has been no drop in performance at all.

Having said all that it is good that windows ships now with a full AV suite as I can remember when you didn`t even get a firewall with it.

Regards Eck

 

[Andy Ful]

Even just knowing that MD uses more resources than some 3rd party AV`s makes me install something lighter as a matter of course.

Normally, Defender does not use more resources than 3rd party AVs. On the contrary, it uses fewer resources than most AVs. The resources are overused occasionally when doing administrative tasks (file management, full scans, etc.). If one uses applications that are doing such tasks in the background then Defender can use more resources. In this case, several 3rd party AVs can use fewer resources.

https://www.mrg-effitas.com/wp-content/uploads/2021/05/MRG_Effitas_360_2021Q1.pdf

 

[Behold Eck]

Normally, Defender does not use more resources than 3rd party AVs. On the contrary, it uses fewer resources than most AVs. The resources are overused occasionally when doing administrative tasks (file management, full scans, etc.). If one uses applications that are doing such tasks in the background then Defender can use more resources. In this case, several 3rd party AVs can use fewer resources.

https://www.mrg-effitas.com/wp-content/uploads/2021/05/MRG_Effitas_360_2021Q1.pdf

I hear what you are saying but I personally wouldn`t be using any of those 3rd party AVs anyway.

I much prefer setups that I know from experiance to be light e.g. C FW, WVStopX and K7 Total Security on different systems.

All down to personal preferance but I just find MD a bit boring really.

Regards Eck

 

[peterfat11]

Is Kaspersky a Waste of Money? NO.

no its not it got more advanced and helpful tools than WD

 

[Evjl's Rain]

Normally, Defender does not use more resources than 3rd party AVs. On the contrary, it uses fewer resources than most AVs. The resources are overused occasionally when doing administrative tasks (file management, full scans, etc.). If one uses applications that are doing such tasks in the background then Defender can use more resources. In this case, several 3rd party AVs can use fewer resources.

https://www.mrg-effitas.com/wp-content/uploads/2021/05/MRG_Effitas_360_2021Q1.pdf

if I see these tests of WD/MD have file copying speed is unchanged compared to no AV, I know they don't test correctly
for example, on the test you posted, the copy operation only lasts 2 seconds and the extraction lasts 6.5 seconds without AV. That's too short to correctly evaluate performance. At least, something between 30s-2mins. In AV-test, MD doesn't even impact the copying speed so how did they do that? They copied files from D: to C:? Potentially flawed

MRG has one of the best malware tests but their performance test is one of the most unhelpful I've ever seen, next to AV-test

 

[Andy Ful]

I hear what you are saying but I personally wouldn`t be using any of those 3rd party AVs anyway.

I much prefer setups that I know from experiance to be light e.g. C FW, VWStopX and K7 Total Security on different systems.

All down to personal preferance but I just find MD a bit boring really.

Regards Eck

Yes. These solutions will be very light.

 

[Andy Ful]

if I see these tests of WD/MD have file copying speed is unchanged compared to no AV, I know they don't test correctly
for example, on the test you posted, the copy operation only lasts 2 seconds and the extraction lasts 6.5 seconds without AV. That's too short to correctly evaluate performance. At least, something between 30s-2mins. In AV-test, MD doesn't even impact the copying speed so how did they do that? They copied files from D: to C:? Potentially flawed

MRG has one of the best malware tests but their performance test is one of the most unhelpful I've ever seen, next to AV-test

All tests (also MRG Effitas) show that Defender is not fast when doing file management. I tested it by myself, too. File copying (EXE, MSI) was about 50% slower compared to no-AV system.

 

[Evjl's Rain]

All tests (also MRG Effitas) show that Defender is not fast when doing file management. I tested it by myself, too. File copying (EXE, MSI) was about 50% slower compared to no-AV system.

I agree. However, I just wanted to point out the performance test of MRG is poorly conducted to due to the short duration
It seems AV-C has the best performance test procedure. However, I also disagree with their tests for some specific AVs, particularly Kaspersky. Kaspersky, IMO, is not that fast amd doesn't deserve that high rank in performance. It slows down copying speed similarly to MD, but a little bit faster. It also slows down web browsing speed and comsumes more CPU during surfing more than other AVs I have used

 

[Andy Ful]

I agree. However, I just wanted to point out the performance test of MRG is poorly conducted to due to the short duration

They probably changed the presentation method from showing the total time (like in Q2 2020) to the average time (total time / number of files).

It seems AV-C has the best performance test procedure. However, I also disagree with their tests for some specific AVs, particularly Kaspersky. Kaspersky, IMO, is not that fast amd doesn't deserve that high rank in performance. It slows down copying speed similarly to MD, but a little bit faster. It also slows down web browsing speed and comsumes more CPU during surfing more than other AVs I have used

The correct test should include many factors. Some important factors are skipped in any performance test I have seen. For example, the AV-Comparatives skipped some factors present in the MRG Effitas test and vice versa.
It is hard to obtain similar results as in the AV testing labs. They use different testing criteria and different hardware. From the fact that AV-Comparatives results fit well to the results obtained on our machines, it does not follow that other tests are faulty. The same can be seen when reading the posts of users on many forums.