User Feedback Microsoft Defender 6/12/2020 Review

[SeriousHoax]

In conclusion: the Achilles heel of Defender is java.
Nothing to worry about when you don't have java installed or use the great tools of @Andy Ful

I think most AV struggle against java based malware except maybe Kaspersky pre-execution and post execution combined.
Like you said, not installing Java and using @Andy Ful's tools is the best way to stay protected.

 

[Gandalf_The_Grey]

I think most AV struggle against java based malware except maybe Kaspersky pre-execution and post execution combined.
Like you said, not installing Java and using @Andy Ful's tools is the best way to stay protected.

That's why with his KIS review it would be interesting to see if KSC Free performs the same against java based malware.

 

[Andy Ful]

It would be interesting, but generally, there is no reason for the advantage to fight JAR malware. There is no AMSI for these files, so the proactive detection is based on heuristics & ML. The advantage over other AVs could be related to detonation in the cloud sandbox, but I did not hear that any AV could do it.
One can avoid Java installation, but it can be easily downloaded/installed by an innocent script (seen in the wild). These problems are similar to Python malware or any malware related to other programming languages.
For now, the best method is script whitelisting (including the JAR and other scripts).

 

[Andy Ful]

The best thing that Microsoft could do for home users, would be to add the script protection option to Windows Home. When enabling this option, the scripts would be analyzed in the cloud with a low "suspiciousness threshold level". This would also produce many false positives for applications that use scripts (Java applications, etc.) and would force the application vendors to whitelist their applications/updates for home users in Microsoft (it takes 5 minutes).

 

[McMcbrad]

it might be very hard to detect some Java malware, as it has virtualisation detection.
This will bypass dynamic analyses, obfuscation will bypass static.
Especially if behaviour is event-triggered (such as push message from C&C server), it might be impossible.
I’ve seen Java malware built just by using innocent GitHub libraries and even I was wondering whether it’s malicious or not...

 

[Andy Ful]

Microsoft Defender with ASR rules was tested on MRG Effitas from Q4 2019. This configuration is similar to ConfigureDefender MAX Protection Level.
Microsoft Word - MRG_Effitas_2019Q4_360.docx (mrg-effitas.com)
Microsoft Word - MRG_Effitas_2020Q1_360.docx (mrg-effitas.com)
Microsoft Word - MRG_Effitas_2020Q2_360.docx (mrg-effitas.com)

MRG Effitas 360 Assessment & Certification Q4 2019 - Q2 2020 (In the Wild + PUA + Ransomware + Financial)
Symantec, Kaspersky, and Eset were slightly tweaked.

--------AV vendor-------------------------------------- Missed samples
Eset Endpoint Security.................................=2(3)
Kaspersky Small Office Security..................=*2(3) <----- corrected
Microsoft Defender......................................=3(2)
Symantec Endpoint Protection Cloud.........=5 <----- corrected missed PUA
Bitdefender Endpoint Security.....................=2(4)

Avast Business Antivirus.............................=7(13)
F-Secure Computer Protection Premium...=22(11)
McAfee Endpoint Security...........................=19(22)
Avira Antivirus Pro.......................................=27(22) <----- corrected missed PUA
Trend Micro Security...................................=65(7) <----- corrected missed PUA

The numbers in the brackets are the samples missed but blocked in 24 hours. Kaspersky did not participate in the last test, but I added the average from other tests.
The first 5 AVs have got scorings that cannot be differentiated by these tests.

Edit.
Sorted by the total number of missed samples. Corrected the testing period to Q4 2019 - Q2 2020.
Added full product names. Corrected Avast data.

Edit2
Skipped the Fileless and Banking protection tests because they are not important in the home environment.
Microsoft Defender free (even with ConfigureDefender MAX settings) cannot be directly compared with the AV Business versions because it does not have Password Manager, VPN and Banking protection.

 

[McMcbrad]

Microsoft Defender with ASR rules was tested on MRG Effitas from Q4 2019. This configuration is similar to ConfigureDefender MAX Protection Level.

MRG Effitas 360 Assessment & Certification Q4 2019 - Q3 2020 (In the Wild + PUA + Ransomware + Financial)
Symantec, Kaspersky, and Eset were slightly tweaked.

------------------------------------ Missed samples
Symantec...................=4
Eset............................=2(3)
Microsoft Defender...=3(2)
Bitdefender................=2(4)
Kaspersky..................=3(3)

Avast..........................=7(10)
F-Secure....................=22(11)
McAfee......................=19(22)
Avira..........................=23(22)
TrendMicro...............=64(7)

The numbers in the brackets are the samples missed but blocked in 24 hours. Kaspersky did not participate in the last test, but I added the average from other tests.
The first 5 AVs have scorings that cannot be differentiated by these tests.

Edit.
Sorted by the total number of missed samples.

Symantec Endpoint Protection, unlike Norton, uses Early Warning Service and these results I am willing to accept (unlike AV-Test and AV-Comparatives). This more or less matches my observations, except Trend Micro that performs bad, but not rock-bottom on my tests. Avast is also better than Bitdefender on my tests.

But following factors are added after edit and worth mentioning:
I have submitted threats to Trend Micro (keeping them on my desktop and rescanning them every day) and it has taken sometimes 3-4 days. In contrast, Defender and Avast auto-get them (I don't even submit anything) and they are gone in minutes.
I've seen reaction time of few hours from ESET.
Bitdefender normally takes a day or two to issue a definition.

This information matters on certain types of threats, such as keyloggers.

 

[geminis3]

Since I'm also a Linux user I prefer to keep things minimal and sticking to the OS builtin AV it's the way to go for me when booting into Windows.

 

[Andy Ful]

From my analysis of AV tests (Consumer tests of AV-Comparatives, AV-Test, SE Labs) in the period of two last years (2019-2020) it follows that the strongest AV protection (Home versions, default settings) available for home users should provide Norton LifeLock.
Today, I have made one day test of false positives related to Norton and Windows Defender with ConfigureDefender MAX settings (without Controlled Folder Access).

I downloaded the fresh-uploaded applications (EXE and MSI, mostly from today) and tried to run them. Here are the results:

15 applications were allowed by Smartscreen Application Reputation & Windows Defender-MAX & Norton:
GOG Galaxy 2.0.30.20 Beta
TraceRouteOK 2.31
Cyotek WebCopy 1.8.2 Build 740
What Watch 4.1 Build 105
Java editor 1.12
Kate 20.12.0 Build 1110
Minsky 2.21.0 Beta 19
R for Windows 4.0.3
Manager (Desktop Edition) 20.10.91
PCSX2 1.6.0
CMake 3.19.1
Java OpenStreetMap Editor 17329
tinyMediaManager 4.0.6
Windows and Office ISO Downloader 8.45.0.152
Portable VidCoder 5.21
Bacula 9.6.7 (blocked by SmartScreen Application Reputation, but allowed by Norton and WD)

19 applications were blocked by SmartScreen Application Reputation, or Windows Defender, or Norton:
jaBuT 2020.12.31850..................................................4000 downloads..(bS, bN, bD)
GPAC 1.1.0 rev 359 DEV........................................10000 downloads..(bS, bN, nbD)
Tablacus Explorer 20.12.11....................................3000 downloads..(bS, nbN, bD)
ThunderSoft Free Flash SWF Downloader 3.2.0.....3000 downloads..(bS, nbN, bD)
Google2SRT 0.7.10 Beta.......................................31000 downloads..(bS, bN, nbD)
PdfScanManager 1.22................................................1000 downloads..(bS, bN, removedD)
CornerFix 1.6.0.2.....................................................7000 downloads..(nbS, removedN, nbD)
Apache OpenOffice SDK 4.2.0 .................................15000 downloads..(bS, bN, bD)
Chromium 89.0.4355.0.........................................1500000 downloads..(nbS, bN, bD)
USB Device Tree Viewer 3.4.4...................................14000 downloads..(bS, bN, bD)
QOwnNotes 20.12.5 Build 796.................................11000 downloads..(bS, bN, bD)
Alternate Archiver 4.110..........................................5000 downloads..(bS, nbN, bD)
Jamulus 3.6.2..............................................................3000 downloads..(bS, bN, bD)
InventoryPlus 2.0.2.2..................................................4000 downloads..(bS, bN, bD)
Syncplay 1.6.7.............................................................2000 downloads..(bS, bN, bD)
Vim 8.2.2135.............................................................59000 downloads..(bS, bN, bD)
Process Hacker Portable 2.39.124........................21000 downloads..(nbS, removedN,nbD)
Batch SlideShow Creator Lite 1.70.0.0....................1000 downloads..(bS, bN, nbD)

nbS - not blocked by SmartScreen Application Reputation
bS - blocked by SmartScreen Application Reputation
nbN - not blocked by Norton
bN - blocked by Norton (with a recommendation to not run the file)
removedN - file removed by Norton
removedD - file removed by Defender
nbD - not blocked by Defender
bD - blocked by Defender

CONCLUSION
From 34 legal (fresh) applications:

1. 18 were allowed by SmartScreen Application Reputation (on execution).
2. 18 were allowed by Norton.
3. 20 were allowed by WD-MAX settings.
4. 15 applications were allowed both by Norton and WD-MAX settings.
5. Norton removed 2 legal applications and WD-MAX removed one legal application.

Now I understand why Norton has so good results in the AV tests. These results come from the Norton Threat Insight (heuristics + file prevalence & reputation). In the WD_MAX settings, similar protection comes from ASR rules (especially from the rule "Block executable files from running unless they meet a prevalence, age, or trusted list criteria").
From my observation, the WD block based on file prevalence, age, and reputation is released in most cases after two days.

 

[McMcbrad]

@Andy Ful , you are absolutely right, but the explanation on the good Norton score is something I mentioned many times on many different threads. The technology is called Norton Insight, Threat Insight refers to the detailed window that shows information on the detected threats.
File Insight refers to the information you can see for any file of interest (right click a file and chose Norton-> File Insight).
Performance Insight refers to the detailed information of what’s slowing you down, which can be found under the performance section (I believe the button is named “Performance History”, but I might be wrong).

So the Norton Insight reputation system is very aggressive towards brand new files and if they are not class-3-signed, most of the time they are removed straight away. Sometimes users are warned not to open this file “until more is known”, but then SONAR (Symantec Online Network for Advanced Response) or simply called their behavioural blocker, counts this as a threat sign, which increases the overall probability of maliciousness. Very few signs will be needed to deem the file as a threat.

 

[Andy Ful]

@Andy Ful , you are absolutely right, but the explanation on the good Norton score is something I mentioned many times on many different threads. The technology is called Norton Insight, Threat Insight refers to the detailed window that shows information on the detected threats.
File Insight refers to the information you can see for any file of interest (right click a file and chose Norton-> File Insight).
Performance Insight refers to the detailed information of what’s slowing you down, which can be found under the performance section (I believe the button is named “Performance History”, but I might be wrong.)

So the Norton Insight reputation system is very aggressive towards brand new files and if they are not class-3-signed, most of the time they are removed straight away. Sometimes users are warned not to open this file “until more is known”, but then SONAR (Symantec Online Network for Advanced Response) or simply called their behavioural blocker, counts this as a threat sign, which increases the overall probability of maliciousness. Very few signs will be needed to deem the file as a threat.

I know. I used the term "Threat Insight" to narrow the Norton Insight to information about blocked/removed files. Generally, one can use "File Insight", "Download Insight", "System Insight", etc., to get information about files/processes.

 

[McMcbrad]

I know. I used the term "Threat Insight" to narrow the Norton Insight to information about blocked/removed files. Generally, one can use "File Insight", "Download Insight", "System Insight", etc., to get information about files/processes.

The only way to bypass Norton (considering you are working only with executables and don’t care about scripts + everything is patched) will be to sign the file with a class 3 signature. This is not handed out like candy, unlike EV and will have to be stolen from a very reputable company. This scenario is highly unlikely and even then it won’t cover the age and prevalence requirement. To get the full Norton trust, a file should be a month old, have more than 10k users and a class-3 signature. These files are exempt from scanning and behavioural monitoring.

I believe Microsoft tactic is similar.

 

[Andy Ful]

The only way to bypass Norton (considering you are working only with executables and don’t care about scripts and others) will be to sign the file with a class 3 signature. This is not handed out like candy, unlike EV and others, and will have to be stolen from a very reputable company. This scenario is highly unlikely and even then it won’t cover the age and prevalence requirement. To get the full Norton trust, a file should be a month old, have more than 10k users and a class-3 signature. These files are exempt from scanning and behavioural monitoring.

I believe Microsoft tactic is similar.

I think, that it can be also bypassed by using an archive (on the flash drive) with legal EXE (vulnerable to DLL hijacking) + malicious DLL. But, I did not test this. Such a technique would probably be prevented (by Norton Download Intelligence) when the file is downloaded from the Internet.

 

[McMcbrad]

I think, that it can be also bypassed by using an archive (on the flash drive) with legal EXE (vulnerable to DLL hijacking) + malicious DLL. But, I did not test this. Such a technique would probably be prevented (by Norton Download Intelligence) when the file is downloaded from the Internet.

I also haven’t tested it, but in the early versions of SONAR, Symantec focused on non-process threats and dlls are also covered by reputation. I haven’t tested process hollowing either, but I don’t think this will work.

Your easiest bypass is to use a script *.ps1 where you have smuggled the payload and you use hackitup or any of the available methods to perform code injection. In that case Norton will see a perfectly harmless executable and won’t raise any alerts. However there you’ll need to find a code injection technique that is still not covered and will have to obfuscate the code to evade detection of the payload itself (if you are smuggling an old threat).
You will have to test various obfuscation and compression methods to find one that won’t trigger packer heuristic.
You will have to test different PS attributes, as these can’t be concealed and no visible window can trigger SONAR.

Another possibility to bypass it is to use Java malware, which is extremely poorly covered in Norton.

Other techniques might also be available, but they will require too much effort.

 

[Andy Ful]

I also haven’t tested it, but in the early versions of SONAR, Symantec focused on non-process threats and dlls are also covered by reputation. I haven’t tested process hollowing either, but I don’t think this will work.

I have been focused on bypassing the reputation feature. SONAR will check DLL but this check will not be so strong without reputation information about PE file.

 

[McMcbrad]

I have been focused on bypassing the reputation feature. SONAR will check DLL but this check will not be so strong without reputation information about PE file.

They treat dll as PE. They don’t differ them and they don’t look for MOTW. The class 3 signature requirement + the age and prevalence still apply. You can smuggle your dll in a script and inject it. You can convert it to an exe, convert it to hex code or base-64 or something and run it through PowerShell. You can then create a scheduled task with that if you want to survive a reboot.

 

[Andy Ful]

They treat dll as PE. They don’t differ them and they don’t look for MOTW. The class 3 signature requirement + the age and prevalence still apply. You can smuggle your dll in a script and inject it. You can convert it to an exe, convert it to hex code or base-64 or something and run it through PowerShell. You can then create a scheduled task with that if you want to survive a reboot.

You are probably right, but I would test it anyway. I noticed that some AV advanced features can detect DLL when executed (for example by rundll32.exe), but not detected when the DLL is loaded by an EXE, especially when the DLL mimics one of the system libraries. Another problem is related to .NET DLLs.

 

[McMcbrad]

You are probably right, but I would test it anyway. I noticed that some AV advanced features can detect DLL when executed (for example by rundll32.exe), but not detected when the DLL is loaded by an EXE, especially when the DLL mimics one of the system libraries. Another problem is related to .NET DLLs.

This will not be a very huge issue, as in a home environment, attackers will only go as far as passing a DLL to rundll.exe or something of this sort. They will rarely go further than this, but it's still interesting to test.

I find working with all sorts of fileless threats easier for a not-too-professional programmer like me, as they revolve around few concepts. In the same time, it's far more effective in bypassing security. I believe attackers feel the same way

 

[Andy Ful]

This will not be a very huge issue, as in a home environment, attackers will only go as far as passing a DLL to rundll.exe or something of this sort. They will rarely go further than this, but it's still interesting to test.

I find working with all sorts of fileless threats easier for a not-too-professional programmer like me, as they revolve around few concepts. In the same time, it's far more effective in bypassing security. I believe attackers feel the same way

I installed Norton 360 once again and checked it when using ProcessHacker (NIghtly version) from the flash drive. After opening the application folder nothing happened. But, on execution Norton used Download Insight (????) for the EXE and all DLLs - the execution was suspended. I excluded the application EXE and it was allowed to run but without Internet access and DLLs were not loaded. After a few minutes, all DLLs from the application folder were removed.
I also tested the ProcessHacker installed on the system disk (before installing Norton 360). In this case, it could be run without any sign from Norton (DLLs were not removed).
So, Norton can also prevent DLL hijacking (for standard DLLs) via Download Insight when DLL is dropped to disk or loaded from the flash drive.
The only unknown is if it can do it also for .NET DLLs - it is not trivial because .NET DLLs are usually compiled on the fly.

 

[McMcbrad]

I installed Norton 360 once again and checked it when using ProcessHacker from the flash drive. After opening the application folder nothing happened. But, on executionNorton used Download Insight (????) for the EXE and all DLLs - the execution was suspended. After a few minutes all DLLs from the application folder were removed.
I also tested the ProcessHacker installed before installing Norton 360. In this case, it could be run without any sign from Norton.
So, Norton can stop DLL hijacking (for standard DLLs) also via Insight. The only unknown is if it can do it also for .NET DLLs - it is not trivial because .NET DLLs are usually compiled on the fly.

Very interesting. It looks like Norton is a bloodhound on DLL and exe files. I will test the code injection method. Downloading Norton now.