App Review Windows Defender vs Top 200 Ransomware (PC Security Channel)

[KnownStormChaser]

Content source

https://www.youtube.com/watch?v=oTRJNfjh_iU

 

[Bot]

Thanks for sharing this video! It's a great resource for anyone interested in understanding the effectiveness of Windows Defender against top ransomware threats. Let's discuss!

 

[Dave Russo]

Leo seems to be a broken record, loves to bash Microsoft Defender, how about some balance showing recommendations, downloading 200 viruses on a clean computer , no backup Leo? Who is paying for your advice? commission? Support chat? Leo is it really a tradition to check Microsoft Defender each year okay then Merry Christmas

 

[Khushal]

Hehe Leo vs Windows Defender again.

 

[bazang]

Leo is it really a tradition to check Microsoft Defender each year okay then Merry Christmas

Yes. Leo does not like Microsoft Defender because for unmanaged Windows for Home and Pro, the Microsoft Defender default configuration is insecure. Plus Controlled Folders protections are not that great.

Backups are not relevant to how Microsoft Defender performs. They are a "compensating control" and the fact that such controls exist, does not forgive Microsoft for doing such a substandard job with Microsoft Defender on unmanaged, home systems.

Windows security has and always be designed and intended to be managed by an administrator or cybersec professional that uses a full-stack of Zero Trust and Defense-in-Depth.

Home users using unmanaged Windows for Home or Windows Pro are given an insecure-by-default operating system. At least the Pro users can harden their OS. And they should because Defender without default-deny is not enough.

Microsoft Defender will always perform poorly in behavioral tests. Its capabilities in this area are not adequate.

Hehe Leo vs Windows Defender again.

Hehe people who disagree vs Leo again.

I guess I will have to explain what Leo demonstrated using VMRAY, which is despite the new ransomware using most of the typical ransomware behaviors, Microsoft missed it BECAUSE its behavioral blocking capability is weak. This is well known amongst those in the know, but obviously completely unknown and misunderstood on security forums.

Microsoft Defender by itself is not enough. Now whether people believe it is equal to other security software, well - they are entitled to whatever they want to think or believe. Leo does not care if people use Microsoft Defender. Why would he? It's not his system. He makes a lot of money from people that use Microsoft Defender and it fails, and then call his company to figure out what went wrong, and fix the problem for the user.

 

[oldschool]

Blah, blah, blah ....

 

[Andy Ful]

If we forget about invalid testing methodology, this video can be an interesting demonstration of (possibly) new ransomware. The sample missed in the test was submitted to Virus Total one day before the test and was poorly detected by other AVs. Interestingly, in the test, it managed to encrypt only some files. We cannot exclude the possibility that the ransomware was detected post-execution (that is how "Block at first sight" works in Microsoft Defender for some unknown samples).
Generally, I like this video demonstration (not a real test). The author tried to be neutral in his comments.

Microsoft Defender missed 1 per 98 samples (the "test" was unfinished) which could often happen for any popular AV. So, I cannot say that this demonstration bashes Microsoft Defender.

The detection of such samples in malware tests mainly depends on which AVs first encountered the sample in the wild. Many AVs can fight such malware similarly to Microsoft Defender (the first machine is infected, but others are protected after some minutes). Any two AVs can often miss different samples - we cannot assume that all samples detected by one AV will also be detected by another AV. That is why in good tests, several AVs should be tested at the same time on the same sample pool. Furthermore, for popular AVs, even 1000 fresh samples can be insufficient to get statistically significant differences.

Edit.
Microsoft recommends increasing the Cloud Protection Level and enabling ASR rules to maximize anti-ransomware protection.

 

[Brahman]

Who on earth with a sane mind does this to his/her production machine? I think nobody. The test he does has no value in the real world unless he downloads each and every file in real time and then try to execute it. 99% of such files will get blocked by the browser alone or the protection employed by adblocker/pup blocker extensions most employs. Its not so nice to see content creators actively misguiding people to believe that Microsoft Defender is no good for a common pc user.

 

[XylentAntivirus]

HydraDragonIOC/PyinstallerWiper at main · HydraDragonAntivirus/HydraDragonIOC Here is the wiper malware which bypassed Windows Defender. It's not Ransomware.

 

[slashingbison]

I’m not sure why there is so much bashing on the comments? I appreciate everyone wil have an opinion on how the tests were carried out but as a high level overview it’s good to see how an AV reacts to threats.

For me it’s very useful.

 

[Andy Ful]

HydraDragonIOC/PyinstallerWiper at main · HydraDragonAntivirus/HydraDragonIOC Here is the wiper malware which bypassed Windows Defender. It's not Ransomware.

Any AV on default settings misses many malware samples every day in the wild. It is not so visible in the malware tests because most of those samples are recognized by AVs as malicious before tests are done.
By the way, PyinstallerWiper changes the file extension of corrupted files to .mlbo, just like the ransomware in the video. Do you know something more about such malware used in the wild?

 

[Andy Ful]

I’m not sure why there is so much bashing on the comments? I appreciate everyone wil have an opinion on how the tests were carried out but as a high level overview it’s good to see how an AV reacts to threats.

For me it’s very useful.

There are 11 posts (comments) so far. Five posts were neutral (including this one). Two posts were very critical. Two posts were positive. Two posts were too short (possibly not positive, but not bashing too). If we take an average, this will be probably the right scoring for the video.

 

[RansomwareRemediation]

I don't see errors in the test. YouTuber tests are the ones that most closely resemble reality, not like laboratories, where you never see the test, only numbers and percentages.
Everyone who understands computing knows that Windows Defender is a mediocre antivirus. Otherwise third-party antiviruses would not exist and we would all use Defender.
Defender is one of the easiest AVs to bypass, and it is totally useless if you disable cloud protection. I have seen it in many tests (even from AV Comparatives itself), the AV's offline protection is considerably mediocre, bordering on poor, if you deactivate its cloud. As they say, there is no worse blind person than someone who doesn't want to see.
Greetings.
pd: There is even malware that bypasses defense and gets into the antivirus exclusions.
The way defender removes malware seems pretty bad to me, once it has penetrated the system.

 

[Andy Ful]

I don't see errors in the test.

It is possible that the term "test" means something else to me and you. So, I can see some obvious errors. For example, the test results should not depend on the order of executed samples. Also, the test with 200 samples, should not end after executing half of the available samples. I do not think that Leo intended to do a real test. He was simply curious if Microsoft Defender could protect against all gathered samples.

In the video, Leo demonstrated that after running (one by one) about 100 fresh ransomware samples Microsoft Defender's default protection was bypassed. He did not test other AVs on those samples, so we cannot relate the result to other AVs. It is impossible to conclude from this test if the result is good or not.

As a test, the video is pretty much useless. Most MT readers already know that any AV can be often bypassed after running 100 fresh malware samples.
As a demonstration, it can be useful for some users (it was useful for me):

App Review - Windows Defender vs Top 200 Ransomware (PC Security Channel)

malwaretips.com

Post edited/updated.

 

[Andy Ful]

Why can the order of executed samples destroy the AV scoring in Leo's test?

If the first sample was the "AV challenge", the test result would be 200 missed malware per 200 samples, even if all samples except the first were detected in another test without the "AV challenge" sample. A similar problem can arise if some samples can weaken the AV & Windows protection, without serious infection. If so, the next samples would have more chance of infecting the system. A clear example would be malware that managed to add folder exclusion in Microsoft Defender but was killed before doing more damage.

 

[simmerskool]

I don't see errors in the test. YouTuber tests are the ones that most closely resemble reality, not like laboratories, where you never see the test, only numbers and percentages.
Everyone who understands computing knows that Windows Defender is a mediocre antivirus. Otherwise third-party antiviruses would not exist and we would all use Defender.
Defender is one of the easiest AVs to bypass, and it is totally useless if you disable cloud protection. I have seen it in many tests (even from AV Comparatives itself), the AV's offline protection is considerably mediocre, bordering on poor, if you deactivate its cloud. As they say, there is no worse blind person than someone who doesn't want to see.
Greetings.
pd: There is even malware that bypasses defense and gets into the antivirus exclusions.
The way defender removes malware seems pretty bad to me, once it has penetrated the system.

3d-party AV have been around since 1980s (IIRC) and windows did not have a good AV in past, but MS Defender has become strong enough in the past year or two as seen in @Shadowra's recent test and other online tests.

 

[Andy Ful]

3d-party AV have been around since 1980s (IIRC) and windows did not have a good AV in past, but MS Defender has become strong enough in the past year or two as seen in @Shadowra's recent test and other online tests.

The results usually depend on the applied testing methodology and can hardly be compared with each other. All tests are only an imperfect simulation of in-the-wild reality. We cannot be sure which simulation is closest to reality. It is good to take into account all sensible tests. Even "tests" (demonstrations) similar to the video in OP can have some value.
I prefer tests made by AV-Comparatives, AV-Test, and SE Labs because they are done periodically for years with accepted methodology (AMTSO), and the results are statistically evaluated (included in AV Awards).

 

[bazang]

If we forget about invalid testing methodology

Leo is not a professional tester. As he has stated publicly he has no interest in performing tests that others find acceptable. His position is that his demonstrations prove whatever point that he is making.

He is correct in that Defender has one of the weakest behavioral detection capabilities.

However, he also never discusses that Microsoft - unlike other security software publishers - provides a full-stack of defense-in-depth security features. Why does he not discuss this? I already know from talking to him. His view and testing is from the standpoint of a n00b who knows nothing about security with Defender at 100% default settings on Windows Home.

As we all know, Microsoft built Windows for enterprises, governments and other institutional type organizations where the endpoints are managed. Microsoft only offers Windows for Home (and therefore default Defender) as it does not want to develop a home-user specific version with better security.

To Microsoft's credit has tried to better protect consumers many times. When it did offer a great improvement in home user security using S Mode, most every "user that wants to use stuff" complained.

You always default to "Most MT members...understand" and I get that, but with utmost respect I disagree. This forum is a public one and for every regular MT member there are probably 100 lurkers. So one cannot justify the interpretations of others based upon them being an MT member alone.

Leo, and most people who are unbiased and honest, know that Defender in its default configuration on Windows Home is not enough. It is not very good insurance even for those who are careful with their online and system usage. Absolutely not good enough for a household with children or other negligent, prolific downloaders. As you point out, Microsoft recommends that certain other features be enhanced so that default Windows provides better protections. It is buried infos that most home users will never look for nor find if they have the inclination to look for it.

My own personal observations of years of testing is that all AV are weak with lots of limitations. They all need "compensating controls" which are unpopular or unacceptable to "users that want to use stuff." I think those people are fundamentally the problem. Until those people get "fixed" by taking away their ability to do what they want, nothing will ever change.

It is also astonishing that people invest so much of themselves both mentally and emotionally in products (fanbois). Any person with common sense understands that they cannot trust any software or digital system, no matter what software is used or how it is configured. They should have zero personal investment in any software or configuration. But that is not how people are. Instead they get upset because someone shows a thing or two about a product, and they don't agree with the review. This is one of the most common social media diseases. It infects the masses across the entire digital space.

 

[Andy Ful]

He is correct in that Defender has one of the weakest behavioral detection capabilities.

I do not remember him saying so or proving it. Can you show an example? Did you mean offline behavioral detection?

 

[bazang]

I do not remember him saying so or proving it. Can you show an example? Did you mean offline behavioral detection?

In the video Leo talks at length about the VMRay analysis of the malware. He shows in the analysis report how that malware uses multiple tactics, techniques, and procedures and Microsoft Defender does not detect them.

Leo is good at what he does, but he is not exactly the best at explaining the points he is trying to make. Combine that with social media where people interpret whatever they read, view or hear in ways that are incorrect, and it is no surprise that there are those people who think Leo is the global leader of a conspiracy against Defender and Microsoft. Still others claim he is an agent of a competitor. The negative viewpoints against Leo are unsubstantiated and so ridiculous that nobody with a shred of common sense can take them seriously.

If Leo made his demonstrations in-person where people were in front of him, and they could ask questions, and Leo could explain, then the interpretations made by people would be nothing like they are after watching his videos.

I quickly understood what Leo was saying about Defender's behavioral detections, but certainly most people missed it and just see the video as another "Here we go again... Leo hating on Microsoft Defender."

You and I both know that testing behavioral detections is a complex subject, but there is sufficient proof available that Microsoft Defender behavioral detection is not one of its strengths. As a product feature, the behavioral detection is not the industry leading best nor is it the worst. Against certain types of threats it will be competitive with other solutions, but against other types of threats it will not do well.

The real strength of Defender is that it can be combined with other Windows security features. The HUGE problem with that is that Microsoft intends that for managed endpoints, and not unmanaged home users.

With a few hardening tweaks (primarily default-deny), native Windows security quickly and easily surpasses the protection capabilities of all other default-allow security or security where the user has to make a decision.