Security News AV-Comparatives Real-World Protection Test October 2018

shmu26

Level 85
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Jul 3, 2015
8,150
It's not bashing Windows Defender, it's pointing out facts, as Lockdown mentioned.

I strongly believe, feel free to disagree, Microsoft should focus on stability and bug resolving on Defender before adding more modules such as "sandbox".. Once you've got a stable product with the most reduced ammount of bugs, start developing anti-executable techniques and modules, like the block at first sight function. If they can deal with Application Control modules and similar instead of signatures, I'm pretty sure Windows Defender would be much more efficient.

I just find it amusing how they can't make it stable or at least more usable, being the devs for the OS it's default installed in. Anyways, after 1809 it doesn't suprise me.
I don't get it, what is unstable about Windows Defender?
And what's wrong with improving the "block at first sight" function to include script detection? Isn't that what we were asking for?
I also don't see what's wrong with running WD processes in sandbox, if it protects the system from potential compromise, and doesn't impact performance.

WD consistently performs well in a "typical user" environment, and that's what Microsoft cares about. It is what it is. You can argue that it needs SmartScreen as a crutch, so okay. Windows is an integrated environment, what's wrong with that?
 
5

509322

It's not bashing Windows Defender, it's pointing out facts, as Lockdown mentioned.

I strongly believe, feel free to disagree, Microsoft should focus on stability and bug resolving on Defender before adding more modules such as "sandbox".. Once you've got a stable product with the most reduced ammount of bugs, start developing anti-executable techniques and modules, like the block at first sight function. If they can deal with Application Control modules and similar instead of signatures, I'm pretty sure Windows Defender would be much more efficient.

I just find it amusing how they can't make it stable or at least more usable, being the devs for the OS it's default installed in. Anyways, after 1809 it doesn't suprise me.

1. It doesn't matter... as long as I post it, there are those that insist that I am bashing. I don't think that they even bother to read my posts within the context of an entire thread or subject matter.

2. Microsoft is famous for bugs that never, ever get fixed. Any IT Pro that uses Group Policy, AppLocker, Application Guard, etc knows. There are bugs in WD that have been present since 1511.

3. People want stability and reliability. They will pick that over everything else, every time. Only a small percentage rate other things as more important.
 
Last edited by a moderator:

erreale

Level 9
Verified
Content Creator
Malware Hunter
Well-known
Oct 22, 2016
409
The usual useless test. Today it makes no sense to run tests based on viral signatures. Maybe and I say maybe, it would make a little more sense a test on BB capabilities, a bit like Matousec did long ago to test the capabilities of HIPS.
 

RoboMan

Level 34
Thread author
Verified
Top Poster
Content Creator
Well-known
Jun 24, 2016
2,399
I don't get it, what is unstable about Windows Defender?
And what's wrong with improving the "block at first sight" function to include script detection? Isn't that what we were asking for?
I also don't see what's wrong with running WD processes in sandbox, if it protects the system from potential compromise, and doesn't impact performance.

WD consistently performs well in a "typical user" environment, and that's what Microsoft cares about. It is what it is. You can argue that it needs SmartScreen as a crutch, so okay. Windows is an integrated environment, what's wrong with that?
Starting by fixing malware removal process. Making the "remove threat" click worth something, because lots of times it doesn't even do anything. Plus most of the time it will start "removing the malware" and literally take ages to do it. Sometimes even tell it's removed while the process is still active. Saw it lots of times.

Nothing wrong with block at first sight and sandbox, actually it's great. Just saying that before implementing more modules like these, they should focus on fixing the software.
 

plat

Level 29
Top Poster
Sep 13, 2018
1,793
I wonder how this thread became (for some) simply another opportunity to bash WD

No, not at all. :giggle: At some point, the user has to start shouldering the responsibilities. Remember when Microsoft first introduced Windows 10, the marketing claim was made that it was the most secure operating system? However, at the time, Windows Defender was routinely tested in comparatives with SmartScreen disabled. Why run a comparatives study and claim it's legit when you are manipulating various attributes of the participants? It's a dirty study. You see it sometimes on YouTube when the "tester" is maybe switching off behavior blocker or whatever. At the end of the day, what does the consumer believe? SmartScreen comes enabled. Disable at your (anyone's) peril.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,040
I think that the discussion about Windows Defender capabilities is slightly off topic here, and there are also too many speculations about it. All Windows Defender pros and cons were discussed many times on MT.
The one thing, that we can be sure, is that Windows Defender in AV-Comparatives 'Real World Protection' tests, scores very well (for over a year) and after April 2018 it has significantly improved its detection without user dependent actions.
Anyway, it would be nice to know what exactly the words 'user dependent' means in those tests (not only for WD).
I can repeat that those results can be hardly connected to the real protection of the home users and to the results presented on MT Malware Hub.(y)
 

elquenunca

Level 3
Verified
Dec 23, 2017
138
Nobody here should believe that Windows Defender on default settings is better than most other AVs, only because the results of one AV-Comparatives test. So, let's look at the results from the last 6 months:
Bit Defender 100+100+100+100+100+100 = 100
Microsoft 100+100+99.5 + (0.5)+100+100+99.5 +5 = 99.8 (+0.1 user dependent)
Kaspersky 99+99.5+100+100+99.6+100 = 99.7
Avast 99.5+100+99.5+98.9+99.6+100 = 99.6

Two questions.
Why did not Kaspersky have a good result in May (99%)?
Probably the coincidence or maybe not all modules worked as they should work, after the major update in April (it would not be the first time).

Why Microsoft has consistently good results for some months? It may be a coincidence or maybe it follows from adding scripts and macros to "Block at first sight" feature (introduced in Windows 10 ver. 1607 1803 ). If one would look at the Microsoft results from about a year ago (July 2017 - April 2018) then Microsoft would score at 98.3 (99.9 including user dependent actions).

Anyway, those results have nothing to do with the real protection of many users, because in the real world the users are infected mostly by ignoring AV detection, running cracks or pirated software, etc. Those infection vectors cannot be properly measured by any AV Lab.

Post edited - I used wrongly the ver. 1803 for "Block at first sight".
bitdefender always produces good results but almost always little is said about it
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,040
Are you sure if these are correct?
According to my last test of WD in default settings in August 2018, it had no improvement. Also did a few tests off-screen, WD never reacted to anything that signatures failed to detect, just a few old payloads downloaded by scripts
Block at first sign never works without tweaking
https://malwaretips.com/threads/13-08-2018-19.85938/#post-756888
'Block at first sight' works only for files with MOTW. That is why it works well in AV_Comparatives tests and usually do not work when the malware was extracted from the malware pack.
 

erreale

Level 9
Verified
Content Creator
Malware Hunter
Well-known
Oct 22, 2016
409
'Block at first sight' works only for files with MOTW. That is why it works well in AV_Comparatives tests and usually do not work when the malware was extracted from the malware pack.
5bef2ddb64895610367829.gif
 
Last edited:

Evjl's Rain

Level 47
Verified
Honorary Member
Top Poster
Content Creator
Malware Hunter
Apr 18, 2016
3,684
'Block at first sight' works only for files with MOTW. That is why it works well in AV_Comparatives tests and usually do not work when the malware was extracted from the malware pack.
so that's basically not useful because smartscreen will intercept the file anyway

I think basically WD for windows 10 is the same as itself for windows 8 without tweaking + exploit protection and folder protection

I haven't seen its BB and cloud working
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,040
so that's basically not useful because smartscreen will intercept the file anyway

I think basically WD for windows 10 is the same as itself for windows 8 without tweaking + exploit protection and folder protection

I haven't seen its BB and cloud working
It can be useful, because many people simply ingnore SmartScreen, except when it is set to Block. Furthermore, SmartScreen does not check the *.js and *.vbs scripts, and macros. I do not want to speculate, so maybe I should test it by myself before saying more.(y)
 

notabot

Level 15
Verified
Oct 31, 2018
703

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,040
Thanks! I’m a little confused Is it the case that smartscreen will block unknown executables only if they’ve been downloaded with edge ? My experience is that smartscreen blocks (or asks) anything it doesn’t recognize even if downloaded with eg Chrome
From Downloads and the Mark-of-the-Web :

"Browsers and other internet clients (e.g. email and chat programs) can participate in the MOTW-marking system by using the IAttachmentExecute interface’s methods or by writing the Alternate Data Stream directly. Update: Chrome uses IAttachmentExecute and thus includes the URL information on Windows 10. Firefox writes the Alternate Data Stream directly and thus does not."

Despite the method of adding MOTW, most web browsers add MOTW to downloaded files.
 
Last edited:

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top