Serious Discussion New video about defender on pc security Channel

[Defenderfan]

 

[Bot]

Thanks for sharing this video! It seems to be very informative and helpful for understanding Microsoft Defender better.

 

[Andy Ful]

Although many malwares can be recognized by observing the created processes as presented in the video, many others can infect the system silently in a second. The test ended after running 484 samples, and 18 samples (3.73%) were allowed to run. Without detailed analysis, we cannot be sure how many of those 18 samples could infect the system. Using HitmanPro could help with known samples, but it cannot be used to identify many new malwares. So, we cannot be sure if the detection in the video was decent (as Leo concluded), or not.

This video does not differ from the videos from 10 years ago, and nowadays it cannot be considered a reliable test, but only a kind of presentation of AV capabilities (Leo called it an educational video). This is also true for most such videos with other AVs. By watching such a video, we must accept all shortcomings that make it only a presentation.

I can say, that after accepting all shortcomings, the video shows the results consistent with the professional tests made by (AV-Test, AV-Comparatives, SE Labs, etc.). Also, most of Leo's comments are probably true, although his meaning of behavioral protection is also from 10 years ago (offline, no telemetry, no heuristics, etc.).

Serious Discussion - Best AVs and Worst AVs in Behavioral Health

There are tons of AV tests everywhere. We also have many AV solutions on the market, which makes the choice not easy. Which antiviruses are currently at the forefront of behavioral protection? Which are not at the forefront but will provide at least good protection? And which are so weak that...

malwaretips.com

 

[bazang]

Although many malwares can be recognized by observing the created processes as presented in the video, many others can infect the system silently in a second. The test ended after running 484 samples, and 18 samples (3.73%) were allowed to run. Without detailed analysis, we cannot be sure how many of those 18 samples could infect the system. Using HitmanPro could help with known samples, but it cannot be used to identify many new malwares. So, we cannot be sure if the detection in the video was decent (as Leo concluded), or not.

Leo said that there could be false positives before saying "the detection is decent."

All the malware executed were "new" malware, but old malware was dropped and yet, still, Microsoft Defender allowed malware for which signatures it already has were allowed to execute. He said further that he has seen this happen - malware for which Microsoft already has signatures being allowed to execute - before.

I can say, that after accepting all shortcomings, the video shows the results consistent with the professional tests made by (AV-Test, AV-Comparatives, SE Labs, etc.). Also, most of Leo's comments are probably true, although his meaning of behavioral protection is also from 10 years ago (offline, no telemetry, no heuristics, etc.).

Serious Discussion - Best AVs and Worst AVs in Behavioral Health

There are tons of AV tests everywhere. We also have many AV solutions on the market, which makes the choice not easy. Which antiviruses are currently at the forefront of behavioral protection? Which are not at the forefront but will provide at least good protection? And which are so weak that...

malwaretips.com

He said there is "non-existent behavioral blocking unless things are turned-on in Group Policy."

 

[Andy Ful]

Leo said that there could be false positives before saying "the detection is decent."

Yes, there could be some false positives, and still, we do not know if the detection was decent or not.

All the malware executed were "new" malware.., but old malware was dropped and yet, still, Microsoft Defender allowed malware for which signatures it already has were allowed to execute.

You are wrong. The video only shows that they were dropped, and the dropper probably tried to execute them many times.

He said further that he has seen this happen - malware for which Microsoft already has signatures being allowed to execute - before.

That is true for some PUAs. Microsoft Defender detects them and proposes the recommended actions. This also happened in the video, but Leo ignored those recommendations. Anyway, the parent process constantly tried to run the dropped files but failed (probably due to the signatures).

He said there is "non-existent behavioral blocking unless things are turned-on in Group Policy."

That is why I posted "his meaning of behavioral protection is also from 10 years ago (offline, no telemetry, no heuristics, etc.)."
Furthermore, his statement is incorrect/imprecise - Group Policy is not required (important when using Windows Home).

 

[oldschool]

Leo has an unbelievable obsession with MS Defender, but of course he needs the click-bait $$$.

 

[Andy Ful]

I cannot blame him. Microsoft Defender is the most popular AV, so many such presentations are expected. It is a good presentation. Leo called it an educational video, which seems to be a proper description.

If Leo reads MT posts, I could recommend using clearer comments. He sometimes uses two sentences one after another when the first is related to what happened in the video, but the second presents only his beliefs (unrelated to what happened in the video). That is why some people wrongly think that both sentences follow from the presented results.
An example can be the fragment of the video (6:33 - 6:43). The last words related to behavioral protection do not make sense in relation to what can be seen in the video. The AV allowed the execution of 18 new malware and no malware caused the infection. How could this happen when the "behavioral protection is kind of non-existent"? This would require some additional comments.

 

[bazang]

You are wrong. The video only shows that they were dropped, and the dropper probably tried to execute them many times.

Rewatch the video. Leo shows the malware in the Z:\Shared\Malware folder and looks-up the detections on VirusTotal - two times/twice. It clearly shows that Microsoft has signatures for the malware, and it executed. Leo discusses what happened.

The malware is named "Unicorn" and it clearly has been downloaded in the background by PowerShell and then executed from Z:\Shared\Malware. See 1:40 of the video. Also see 4:30 of the video for the further discussion.

Microsoft already had signatures for "Unicorn-??????" but allowed it to execute. The Unicorn-named files are mostly all 0 KB with every 8th or 10th one being ~ 145 KB to 435 KB. Then there is the "update.exe" file - which was created or dropped, but there is no evidence it executes.

The 0 KB file size can be misleading, as it could be deliberate obfuscation of the actual file size. But there is no doubt that "Unicorn-??????" kept executing over-and-over. We cannot know what it was intended to do - but nevertheless it did execute. There was a GUI element on the Desktop and the file kept replicating itself. So, yes, it was executing on the system. There is no evidence that Microsoft Defender blocked it.

There are viruses out there that will do nothing except keep executing to load data into active memory (RAM) and make the system do many writes to the file system/disk. An attacker that wants to overwhelm a system can do so by having writes of 0 KB files in an endless loop - which is clearly what was happening on the test system until it froze and Leo had to reset the test virtual machine to a prior snapshot.

At 5:25 Leo shows what "Unicorn-??????" does via the VirusTotal sandbox "run" analysis and it clearly shows that it creates copy-after-copy of itself - which is the exact behavior observed on the system.

However, at 6:15 Leo says that it could have been the parent file that executed that created Unicorn-??????, but that is not consistent with what the VirusTotal sandbox analysis clearly shows. So he does not commit to anything other than "Microsoft Defender allowed a malware to execute and it rendered the system inoperable." This statement is 100% accurate.

That is true for some PUAs. Microsoft Defender detects them and proposes the recommended actions. This also happened in the video, but Leo ignored those recommendations. Anyway, the parent process constantly tried to run the dropped files but failed (probably due to the signatures).

During the video not a single Microsoft Security prompt appeared asking the user to make a decision. Every single alert that appears in the entire video from Microsoft Security is "MIcrosoft Defender Antivirus detected threats. Get Details."

There are a few notifications from Microsoft Security instructing the user to restart the system.

So Leo is not ignoring any alerts where he had to make a decision to Allow or Block a program, as there are no notifications asking him to make a decision.

Leo has an unbelievable obsession with MS Defender, but of course he needs the click-bait $$$.

Leo says at 5:35 that Microsoft Defender "detection is not too bad, which is fairly decent considering these are brand new files" and at 6:30 he says "It is not any worse than other AVs in terms of signatures." At 7:00 Leo begins talking very positively about Microsoft Defender.

The criticism of Microsoft Defender's over-reliance upon the cloud, its poor heuristics and behavioral blocking, are accurate and fair criticisms.

I have access to the full E5 Government offering that includes every single part of Microsoft Security from Microsoft Defender to EDR to Sentinel to Compliance. And even with all those Microsoft Security components properly and securely configured there are systems that get breached and infected. It is only after users are prevented from downloading or executing anything - and blocking LoLBIN execution, do the bypasses, infections, and compromises drop-off almost to zero.

If it were Kaspersky or Bitdefender on Leo's same test system, the poor Defender showing is almost certainly would not have been replicated.

For users that know and prioritize security, should that matter? Perhaps, but most of the protections are coming from the user and not the security software. Nevertheless, Microsoft Defender as a standalone security solution is not adequate and sufficient for any household using Windows Home with downloading minions that know nothing about security.

 

[oldschool]

Leo says at 5:35 that Microsoft Defender "detection is not too bad, which is fairly decent considering these are brand new files" and at 6:30 he says "It is not any worse than other AVs in terms of signatures." At 7:00 Leo begins talking very positively about Microsoft Defender.

The criticism of Microsoft Defender's over-reliance upon the cloud, its poor heuristics and behavioral blocking, are accurate and fair criticisms.

All of this is true without any doubt. Nonetheless, he keeps pumping out Defender vids with essentially the same message, over and over. And I agree that one might as well not use any AV at all since they're all equally deficient in the long run.

 

[bazang]

All of this is true without any doubt. Nonetheless, he keeps pumping out Defender vids with essentially the same message, over and over. And I agree that one might as well not use any AV at all since they're all equally deficient in the long run.

Send Leo a "Get Well Soon" card with a note that includes a "mental bandaid" (a real bandaid with the word "Mental" written across it) advising him to take care of his Microsoft Defender obsessions.

I bet he gets a bunch of people who are security enthusiasts that attempt to get ThreatLocker though... which, Leo is not really on-board with default-deny, particularly for home users. He does state that it is for enterprises - emphasizing that a few times. Otherwise he believes - as I do - that incapable users should never be permitted to make a decision. Either the system has to be managed or the user needs to be locked-out of the system. This is where Leo and I differ. He still subscribes to the dinosaur thinking that "users that want to use stuff" should be allowed to do whatever they want.

Leo, like Neil Rubenking, assume and generalize that home users are ignorant, careless, negligent, dumb, stupid, and make bad decisions. Except for that 1%. Leo and Neil are correct except that the 1% can be equally dumb, stupid, and make decisions - in all areas of their lives except, perhaps, security.

Someone could probably pay their monthly bills by posting weekly tests of Microsoft Defender. An influencer whose side-hustle is just chopping-it-up online about Microsoft Defender. Every. Single. Week.

 

[Andy Ful]

@bazang,

Half of your post was unnecessary. It is clear that Leo did some extended analysis for one malware sample (Unicorn), but he did nothing for the rest of the executed samples (17 samples). So, what blocked those samples? We do not know, and neither Leo do.

Let's assume that those 17 samples were blocked similarly to Unicorn (although there is no evidence for that). So, Microsoft Defender detected all malware (including payloads) without behavior blocking, except one sample that was fully mitigated without behavior blocking. So, behavior blocking was not tested at all (not challenged in this presentation). In this way, Leo would be right to conclude that Microsoft Defender has decent signatures, but his statement about behavior blocking has nothing to do with the video.

But, a more probable scenario is that some of those samples were blocked by behavior-based detections (locally or in the cloud).

During the video not a single Microsoft Security prompt appeared asking the user to make a decision. Every single alert that appears in the entire video from Microsoft Security is "MIcrosoft Defender Antivirus detected threats. Get Details."

There are a few notifications from Microsoft Security instructing the user to restart the system.
So Leo is not ignoring any alerts where he had to make a decision to Allow or Block a program, as there are no notifications asking him to make a decision.

I do not agree. As you have noticed, two Microsoft Defender alerts required the user actions (restarting the device). Ignoring those alerts caused the system inoperable (which worried Leo). If Leo did restart the system, Microsoft Defender would kill all unwanted processes without problems.

The criticism of Microsoft Defender's over-reliance upon the cloud, its poor heuristics and behavioral blocking, are accurate and fair criticisms.

There is no evidence for poor heuristics and behavior blocking. Leo is not an expert on Microsoft Defender.

I have access to the full E5 Government offering that includes every single part of Microsoft Security from Microsoft Defender to EDR to Sentinel to Compliance. And even with all those Microsoft Security components properly and securely configured there are systems that get breached and infected.

The same is true for other solutions, except default-deny. So according to your argumentation, all non-block-by-default solutions have poor heuristics and behavior blocking.

If it were Kaspersky or Bitdefender on Leo's same test system, the poor Defender showing is almost certainly would not have been replicated.

Leo did not present poor Defender showing in his videos, except for some videos with disabled Internet.

For users that know and prioritize security, should that matter? Perhaps, but most of the protections are coming from the user and not the security software. Nevertheless, Microsoft Defender as a standalone security solution is not adequate and sufficient for any household using Windows Home with downloading minions that know nothing about security.

Leo and I do not share your viewpoint. In his recent videos, he says the opposite. He sees the difference in such features as VPN, Password Manager, staff support, human expert analysis, etc. However, such features are common in businesses and not at home. Furthermore, Microsoft Defender is not a standalone security solution on Windows 10+. You have also Edge web browser, Windows Firewall, SmartScreen (integrated with File Explorer), Core Isolation, etc.

 

[Defenderfan]

I have a question so have big a difference does Automatic sample submission make?

 

[Andy Ful]

I have a question so have big a difference does Automatic sample submission make?

Enable block at first sight to detect malware in seconds - Microsoft Defender for Endpoint

Turn on the block at first sight feature to detect and block malware within seconds.

learn.microsoft.com

Cloud protection and sample submission at Microsoft Defender Antivirus - Microsoft Defender for Endpoint

Learn about cloud-delivered protection and Microsoft Defender Antivirus

learn.microsoft.com

It is required by "Block at first sight" feature, and Microsoft Defender Cloud protection needs it to work with full strength.

 

[Andy Ful]

Learning from Leo.

Let's suppose that @Andy Ful would like to make an educational video sponsored by a vendor of some default-deny protection (like ThreatLocker from Leo's video). It is easy to show the result of running malware on a computer with default-deny protection - the malware will be blocked by default. If we want to add some educational content, it would be good to show the well-known fact that the standard AV will not do as well as the default-deny solution.

I do not want to manipulate samples, so the number of samples must be large enough to show the AV failure. With the pule of 2000 moderately fresh samples, the Defender's chances of missing 0 samples are sufficiently low (roughly 1%).

Now I can take 2000 fresh samples and run them one by one until Microsoft Defender fails. I do not need to inspect the logs, autorun entries, possible code injections, loaded DLLs, firewall connections and packets, or other IOCs. The educational video must be short and easy to watch, so the details must be skipped. My goal is accomplished when Defender fails once in a visible/obvious way (even if it also failed silently several times). Next, I can run samples against the default-deny to show how it works and blocks by default all executed malware.

Can I use a similar video to evaluate the protection of two standard AVs?
Not really, except if one of those AVs prevents the execution of all samples (100% blocks) and the second fails a few times in a visible/obvious way.
Did anyone have the luck to watch such a video?

Post edited/shortened.

 

[bazang]

Let's assume that those 17 samples were blocked similarly to Unicorn (although there is no evidence for that).

Microsoft Defender allowed Unicorn to run. It is the malware that caused the system to freeze. Microsoft Defender has signatures for it but still it allowed Unicorn to run.

I don't care about anything else and I have not been talking about them.

I do not agree. As you have noticed, two Microsoft Defender alerts required the user actions (restarting the device). Ignoring those alerts caused the system inoperable (which worried Leo). If Leo did restart the system, Microsoft Defender would kill all unwanted processes without problems.

I clearly said "There were no Microsoft Defender alerts requiring the user to make a ALLOW or BLOCK decision."

The system restart notifications are not relevant to what I said, but I did accurately state that two (2) did appear.

There is no evidence for poor heuristics and behavior blocking. Leo is not an expert on Microsoft Defender.

Yeah, actually he is. He knows way more than people here suspect or realize. He is constantly criticized as "not knowing what he is talking about when it comes to Microsoft Defender", but actually, he does - very much so.

He just is not going to produce videos that are acceptable to people here. He creates videos in his own way.

Leo does not believe it is his job to educate users to anything other than the risks of relying upon antivirus software. He is open and honest that he does not like Microsoft Defender because in his extensive testing, it just has never performed well - and he uses Kaspersky as his basis of judging other AV test results. He uses Kaspersky as a standard of comparison - even if he does not explicitly state this in every video. He is not going to provide the amount of detail that some people here demand of him. He has to prove nothing to no one.

The same is true for other solutions, except default-deny. So according to your argumentation, all non-block-by-default solutions have poor heuristics and behavior blocking.

I never argued that "all non-block-by-default solutions have poor heuristics and behavior blocking." I never said any such thing. You are putting words into my mouth.

What I said is until default deny is deployed, system infections remain significantly high but then drop to virtually zero (when proper default deny is deployed and managed by knowledgeable and skilled personnel).

Leo did not present poor Defender showing in his videos, except for some videos with disabled Internet.

I did not say that Leo did. What I said was "If Leo had used Kaspersky or Bitdefender in the video instead of Microsoft Defender, either of those two solutions would not have resulted in a frozen/locked test system." Of course there is no absolute proof of this but the probability is very high that this statement is correct.