App Review Best Antivirus vs Windows Defender: What's the difference? (PC Security Channel)

[bazang]

The detection of 40% to 60% of new malware used in the targeted attack on Enterprises is nothing new.

It is same range for new malware encountered by home users.

Some in government state that all AV is no better than 40% effective regardless of the attack types. The idea being that if one does enough large scale testing, eventually all results will converge on that 40% or lower value.

Given that I routinely deal with networks full of personnel-created holes that are under constant attack, my experience with AV is that they are ALL - every single brand and version - are unreliable. While some will catch attacks that others do not, in the end they are all far less effective than the AV lab test results lead people to believe.

The only effective solutions are system lockdown of all devices including network devices, user lockdown, and disabling of a lot of unneeded or rarely needed OS features. This is the golden rule of blue team operations whether it is Windows, Linux or some other OS.

Nobody is ever going to come up with anything more effective than this. And no, it does not make information systems unusable. They are usable but under tightly controlled policies, procedures and practices. This is how effective cybersecurity works. It is just that home users cannot handle it.

 

[bazang]

Leo Mal X is much better then just dropping malware when maybe some will win based on intelligence of the common malware
So I hope he continues to develop it anyway you're totally correct and it's annoying that labs don't use enough apt tests and even when they do they are mediocre considering it's a research "lab" at least from what's available to the public


Anyway I really hope project zero will go back to doing research into sandboxes (emulation) ,the great research they did showing how the mini filters of avs can be easily waeponized by a talented researcher (worse of all they all run at trusted installer , system and they can be trigger to scan , emulate even by code inside html or scripts aka zero click )

Anyway really interesting what you personally use in the "civilian" life (not an endorsement or anything related to any agency , business rather personal preference)

Leo's presentation style reflects his disregard for what he thinks. He is no a video producer. He just wings it as he goes along.

I carefully listen to what he says and I read nothing into it. Watching Leo's videos, 90% of the people here at MT make assumptions or interpretations that Leo himself never states or utters.

That is the fundamental problem with digital communications and media.

 

[bazang]

Not disputing you, but IIRC companies like AppGuard would advertise the gov't uses our software. Their ads sorta read like a gov't endorsement. My wife was director of local federal gov't office about 15 years ago, but she (now retired) has no recollection of security software they used, although I do recall gov't flying in techs 2 techs to work on their computers, they seemed like serious people.

A software publisher can legitimately state that a government uses their software if that is indeed the case. At least in the EU and similar nations.

However, a government that does internal testing of the software is not ever going to publicly release the test results because poor test results will surely result in claims of poor testing by incompetent government personnel, if not also threats of lawsuits.

If the NSA had publicly released the results of the NSA's FinSpy testing against all the AVs about 10 years back, then all those AVs except Comodo and Emsisoft would have sued the U.S. NSA. But since it was Julian Assange and WikiLeaks that stole and released the infos, none of the AVs could do a thing about it.

 

[bazang]

I'm reporting your right now, because you've made me emotionally and mentally upset. I don't know if I can handle the truth. Time for jisatsu.

This brought on the hard chuckle.

Posting on forums or social media is like a box of chocolates. You never know what you're gonna get.

I truly, and honestly, am amazed that so many people get upset by things posted online. This is why professional trolls earn 100,000+ Euros per year. The troll is not the problem. The problem is the people who are so sensitive and offended over most anything.

 

[Andy Ful]

It is same range for new malware encountered by home users.

This is an interesting statement, but it is unsupported by available sources. One could suspect a global conspiracy to hide the truth.
We cannot discuss this on MT without some proof.
There is also a simple explanation that you might misinterpret the data. Some of the data you are referring to might be static detection or offline detection

Edit.
I assume that by detection we understand the detection with cloud backend support, code emulation, Machine (Deep) Learning, behavior-based detections, AMSI-based detections, post-launch (post-infection detections), etc.

 

[bazang]

Edit.
I assume that by detection we understand the detection with cloud backend support, code emulation, Machine (Deep) Learning, behavior-based detections, AMSI-based detections, post-launch (post-infection detections), etc.

Yes. The detection range is consistent across versions of Microsoft Defender. Actually, you would think that the untweaked, limited feature home version would perform much worse. In reality it is only 5% to 10% less effective at the home user defaults and that is not always consistent.

I will state it this way. Governments know the capabilities of Microsoft Defender (and all other AVs) very well and they do not view them as much of a deterrent. Given that organized crime now has the same level of budgets as governments, what is true of the government is true of organized crime. One has to also consider that governments contract or have off-contract agreements with criminal organizations to further their geopolitical agendas. The benefits to the criminal enterprises is not limited to revenue. They also get access to government-only technologies.

When I have time I can make an image of the government testing workflows that I know of from first-hand experience.

I am not interested in praising or bashing Microsoft Defender (or any other AV). I am only interested in reality. My own personal testing with various lab setups and personally coded malware is that the "You are protected" marketing slogan cannot be trusted absolutely. Default-Allow protections will fail a person or organization when they are needed most. They all are just software with backends. Most users who focus on protecting localhost are very well served by Microsoft's built-in security. There is a lot of it to be had by the user. My greatest wish is that users would be inclined to learn about and use as much of it as they can.

The single greatest threats are users downloading and executing medium-aged to old malware and then clicking on links or navigating to high-risk websites. In most such cases, all the top brand AVs will protect the user sufficiently well. Some might let through PUA/PUPs while others will not. But overall, we could agree that the "You are protected" marketing slogan is mostly accurate for all the leading brand AVs for the average world citizen that is on latest patched Windows Home and they do the typical online content, media & file consumption - Microsoft Defender being among the leaders.

What I think is far, far more important than test results are the little gems of knowledge that you have discovered along your - what? - 8+ year journey with Defender and native Windows security? Your notebook is thick.

Edit: FYI Andy, the old ways are dying. Intimate knowledge of OS security internals is dying and dying fast. Organizations are moving to 100% cloud native all run by AI and maintained by a few hoomans. It is happening so fast that SANS Institute just dropped multiple certifications such as PowerShell security, Exploit mitigation, etc. It is all - and I mean the ENTIRE security stack - is being moved to the cloud and automated & run by AI. I am not sure people understand the significance of SANS Insitute dropping courses, but it is huge. There are unseen SEISMIC shifts happening in the digital workforce. Expect hundreds of thousands to be permanently laid-off or terminated from their current jobs within a few years. This is the reality of the AI security revolution. That NOC or SOC that once had 10 or more people at a larger enterprise now only has 3 or less - or more likely - has been off-loaded/subcontracted to a MSP or MSSP that manages security for 100 or more businesses.

 

[Andy Ful]

When I have time I can make an image of the government testing workflows that I know of from first-hand experience.

It would be interesting to see and analyze them. From my experience, the detection range 40%-60% is rather expected for 0-hour malware.
The high protection rate in the Real-World tests made by AV-Comparatives, AV-Test, and SE Labs follows from the fact that most of the tested samples are probably one or more days old. The delay is related to gathering and analyzing the samples.

 

[Andy Ful]

Here is an old but still interesting article from the signature/heuristics era:

Anti-virus is a Poor Substitute for Common Sense – Krebs on Security

Since the year 2010, both malware and AVs developed much, but the words of David Harley (AMTSO board member and director of malware intelligence for NOD32 maker ESET) about 0-hour malware may still be valid:

I’m not quarreling with the proposition that the industry misses a lot of malware. That’s incontrovertible, when every day we’re dealing with close to 100,000 new malware samples. In fact, that sort of level of detection that NSS is talking about — 50 to 60 percent right out of the gate — sounds realistic to me.

 

[Vitali Ortzi]

Here is an old but still interesting article from the signature/heuristics era:

Anti-virus is a Poor Substitute for Common Sense – Krebs on Security

Since the year 2010, both malware and AVs developed much, but the words of David Harley (AMTSO board member and director of malware intelligence for NOD32 maker ESET) about 0-hour malware may still be valid:

I wonder if Microsoft getting top results in a recent test done here in the forum is mainly due to better intelligence
Although some were blocked based on behavioral monitoring and not just signatures

 

[Andy Ful]

I wonder if Microsoft getting top results in a recent test done here in the forum is mainly due to better intelligence
Although some were blocked based on behavioral monitoring and not just signatures

It is hard to derive conclusions from one test.

 

[Sandbox Breaker - DFIR]

It is same range for new malware encountered by home users.

Some in government state that all AV is no better than 40% effective regardless of the attack types. The idea being that if one does enough large scale testing, eventually all results will converge on that 40% or lower value.

Given that I routinely deal with networks full of personnel-created holes that are under constant attack, my experience with AV is that they are ALL - every single brand and version - are unreliable. While some will catch attacks that others do not, in the end they are all far less effective than the AV lab test results lead people to believe.

The only effective solutions are system lockdown of all devices including network devices, user lockdown, and disabling of a lot of unneeded or rarely needed OS features. This is the golden rule of blue team operations whether it is Windows, Linux or some other OS.

Nobody is ever going to come up with anything more effective than this. And no, it does not make information systems unusable. They are usable but under tightly controlled policies, procedures and practices. This is how effective cybersecurity works. It is just that home users cannot handle it.

Amen

 

[Andy Ful]

Machine learning and code emulation could significantly increase the detection of FUDs. For example:

https://blackhat.com/docs/us-14/materials/us-14-Mesbahi-One-Packer-To-Rule-Them-All-WP.pdf

The chart is from the year 2014. Currently most AVs can probably use code emulation for FUDs.
Unfortunately, the malware evolved to bypass code emulation so we still can see the cat-and-mouse game.

 

[Vitali Ortzi]

Machine learning and code emulation could significantly increase the detection of FUDs. For example:

https://blackhat.com/docs/us-14/materials/us-14-Mesbahi-One-Packer-To-Rule-Them-All-WP.pdf

View attachment 287054

The chart is from the year 2014. Currently most AVs can probably use code emulation for FUDs.
Unfortunately, the malware evolved to bypass code emulation so we still can see the cat-and-mouse game.

Now there is cloud emulation for different av software to increase it further but you it has its own limitation and there is no perfect solution in default allow so even with local , cloud emulation you can't stop everything

https://www.blog.checkpoint.com/security/unlocking-the-power-of-in-context-emulation-in-malware-sandboxing/

 

[Andy Ful]

Interesting statistics:

40+ Antivirus Statistics & Trends (2024)

Discover the latest facts, trends, and statistics in the antivirus industry that are shaping the current state and forecasting the industry’s future…

vpnalert.com

(...)
4. 8.9% of free antivirus users were still affected by virus attacks in the past 12 months. A slightly lesser 7.4% of paid users suffered a similar fate in the same timeframe (2021-2022).
(...)
8. ESET blocked over 93% of targeted attacks in 2022. The antivirus software provider fended off 14 out of 15 targeted attacks to take the top spot.
It is closely followed by Bitdefender (13 of 15 attacks) and G Data/Kaspersky/Microsoft (12 out of 15 attacks).

The difference in protection between paid and free AVs was less than 2%.
In 2022, top AVs blocked more than 80% of targeted attacks.

 

[Jonny Quest]

Interesting statistics:

40+ Antivirus Statistics & Trends (2024)

Discover the latest facts, trends, and statistics in the antivirus industry that are shaping the current state and forecasting the industry’s future…

vpnalert.com

The difference in protection between paid and free AVs was less than 2%.
In 2022, top AVs blocked more than 80% of targeted attacks.

As you said, interesting article (statistics) indeed. And what I also liked, were the 18 references listed at the end of the article, that it just wasn't their research alone.

 

[Andy Ful]

Another one:

Virus Alert: 31 Antivirus Statistics and Trends for 2025

The antivirus software market is still growing. Here’s a roundup of the most recent antivirus statistics and trends in the industry.

dataprot.net

On a monthly basis, only 4% of Windows Defender users encounter a piece of malware.​

(Microsoft)

The adoption rate of Windows 10 has helped the growth of Microsoft’s own antivirus program. The tech giant’s latest report shows that, on average, fewer and fewer users are encountering malware, including the infection attempts that Defender successfully blocks.

So, the average Defender user may expect 1 malware per 2 years. Assuming the 5% infection rate in the wild (10 times higher than in AV testing labs), the average user can be infected 1 time per 40 years. A happy-clicker can probably be infected a few times in a lifetime.

Edit.
Some information is outdated and can be misguiding, like the claim that AVs can detect only 25% of malware. In fact, this information is related to 0-hour malware in 2010 and was discussed in one of my previous posts.