New Update How the hell WD works on Windows Home & Pro?

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176
Managing script false positive detections (antimalware, AMSI, ASR).

Defender can detect/block scripts by several different security layers, for example:
  1. Antimalware (pre-execution) detection.
  2. AMSI-paired machine models (pre-execution) detection.
  3. AMSI-paired machine models (post-execution) detection.
  4. ASR rules.
False detections from points 1, 2, and 3 can be separately excluded via Security Center. The false positive ASR blocks can be excluded by using PowerShell, GPO, or ConfigureDefender.

Please note: Post-execution AMSI-based detections cannot be avoided by adding path exclusions for scripts via Add-MpPreference PowerShell cmdlet (or Defender Policy via GPO)!

AMSI-based detections are similar to other behavior-based detections. AMSI is used to supply machine learning models with code in clear text to avoid string obfuscation.
In the pre-execution case, the code is scanned and the script is blocked if recognized as malicious by behavior models.
In the post-execution case, the code execution is monitored at the runtime and analyzed by dynamic behavior models. The execution is interrupted when the suspicious actions exceed the detection threshold.

Malicious scripts blocked by AMSI-paired machine models are reported in Microsoft Defender Security Center as follows (one year old classification):
  • Trojan:JS/Mountsi.?!ml
  • Trojan:Script/Mountsi.?!ml
  • Trojan:O97M/Mountsi.?!ml
  • Trojan:VBS/Mountsi.?!ml
  • Trojan;PowerShell/Mountsi.?!ml

Some scripts can be blocked by more than one detection.

Microsoft uses many other interesting script detections, for example:
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=Script
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=powershell
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=amsi
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=ams
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=:js/
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=:vbs/
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=bat

Detecting Office macros:
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=w97m
https://www.microsoft.com/en-us/wdsi/threats/threat-search?query=o97m

Post updated.
The problems with whitelisting and exclusions of AMSI-based detections were finally solved by Microsoft:
https://malwaretips.com/threads/how...ipt-as-containing-a-threat.107234/post-935955
 
Last edited:

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176
Some info about training AMSI machine learning models.

"Antimalware Scan Interface (AMSI) helps security software to detect such malicious scripts by exposing script content and behavior. AMSI integrates with scripting engines on Windows 10 as well as Office 365 VBA to provide insights into the execution of PowerShell, WMI, VBScript, JavaScript, and Office VBA macros. Behavioral blocking and containment capabilities in Microsoft Defender Advanced Threat Protection (ATP) take full advantage of AMSI’s visibility into scripts and harness the power of machine learning and cloud-delivered protection to detect and stop malicious behavior. In the broader delivery of coordinated defense, the AMSI-driven detection of malicious scripts on endpoints helps Microsoft Threat Protection, which combines signals from Microsoft Defender ATP and other solutions in the Microsoft 365 security portfolio, to detect cross-domain attack chains.
On endpoints, performance-optimized machine learning models inspect script content and behavior through AMSI. When scripts run and malicious or suspicious behavior is detected, features are extracted from the content, including expert features, features selected by machine learning, and fuzzy hashes. The lightweight client machine learning models make inferences on the content. If the content is classified as suspicious, the feature description is sent to the cloud for full real-time classification. In the cloud, heavier counterpart machine learning models analyze the metadata and uses additional signals like file age, prevalence, and other such information to determine whether the script should be blocked or not.
These pairs of AMSI-powered machine learning classifiers, one pair for each scripting engine, allow Microsoft Defender ATP to detect malicious behavior and stop post-exploitation techniques and other script-based attacks, even after they have started running. In this blog, we’ll discuss examples of Active Directory attacks, including fileless threats, foiled by AMSI machine learning.
"

Diagram showing pairs of machine learning models on the endpoint and in the cloud using AMSI to detect malicious scripts

Figure 1. Pair of AMSI machine learning models on the client and in the cloud

....

"To ensure continued high-quality detection of threats, the AMSI machine learning models are trained per scripting engine using real-time protection data and threat investigations.
Featurization is key to machine learning models making intelligent decisions about whether content is malicious or benign. For behavior-based script logs, we extract the set of libraries, COM object, and function names used by the script. Learning the most important features within the script content is performed through a combination of character ngramming the script or behavior log, followed by semi-asynchronous stochastic dual coordinate ascent (SA-SDCA) algorithm with L1 regularization feature trimming to learn and deploy the most important character ngram features.
On top of the same features used to train the client models, other complex features used to train the cloud modes include fuzzy hashes, cluster hashes, partial hashes, and more. In addition, the cloud models have access to other information like age, prevalence, global file information, reputation and others, which allow cloud models to make more accurate decisions for blocking.
"

"On endpoints, Microsoft Defender ATP uses multiple next-generation protection engines that detect a wide range of threats. One of these engines uses insights from AMSI and pairs of machine learning models on the client and in the cloud working together to detect and stop malicious scripts post-execution.
These pairs of AMSI models, one pair for each scripting engine, are part of the behavior-based blocking and containment capabilities in Microsoft Defender ATP, which are designed to detect and stop threats even after they have started running. When running, threats are exposed and can’t hide behind encryption or obfuscation. This adds another layer of protection for instances where sophisticated threats are able to slip through pre-execution defenses.
"


Edit.
Interesting article about AMSI-based detections (a lot of articles in the bibliography).
 
Last edited:

LaurentG

Level 1
Mar 15, 2021
26
Managing script false positive detections (antimalware, AMSI, ASR).

Defender can detect/block scripts by several different security layers, for example:
  1. Antimalware (pre-execution) detection.
  2. AMSI-paired machine models (pre-execution) detection.
  3. AMSI-paired machine models (post-execution) detection.
  4. ASR rules.
False detections from points 1, 2, and 3 can be separately excluded via Security Center. The false positive ASR blocks can be excluded by using PowerShell, GPO, or ConfigureDefender.

AMSI-based detections are similar to other behavior-based detections. AMSI is used to supply machine learning models with code in clear text to avoid string obfuscation.
In the pre-execution case, the code is scanned and the script is blocked if recognized as malicious by behavior models.
In the post-execution case, the code execution is monitored and analyzed by dynamic behavior models. The execution is interrupted when the suspicious actions exceed the detection threshold. Although the scripting engine alerts can suggest that the particular script code is blocked (like "Wshshell.run"), it does not mean that after excluding this detection, the blocked script code (like "Wshshell.run") will be allowed for other scripts. It is an important feature because otherwise, such exclusions would decrease the protection.

Hi Andy,
there is something I still do not understand.
I guess you wrote this post following our recent discussion.
And my problem was precisely that I was not able to exclude false detection from point 2 or 3.

While you write it's feasible via Security center.

Since I'm sure you know Defender a lot and a lot better than me, could you please explain in detail how to exclude such detection ?

Thanks in advance
 

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176
Last edited:

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176
Fuzzy hashes as words and "Block at first sight"

I noticed an interesting article about deep learning where segments of fuzzy hashes are used as words. Next, the ML models are trained on these "words" to recognize malicious patterns, similarly to ML models in language recognition. This can help to find the previously undetected files and can be used for malware classification according to type, family, malicious behavior, or threat actor.
From the Microsoft article, it follows that this technique is used in the "Block at first sight" feature. Fuzzy hashes are also used in AMSI-based script detection.

 

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176
Cloud Protection Levels

There is a lot of misunderstanding about how Cloud delivered protection works. Here is a diagram from the Microsoft documentation:


1651941168529.png

The red arrow shows the moment when Cloud protection level "Zero tolerance" (Block setting in ConfigureDefender) will block the unknown file. The decision is made in the cloud after the analysis of telemetry (file metadata check). The file is not uploaded to the cloud. Some files can still infect the system if they are not recognized as suspicious by local protection layers - they are not checked by the cloud backend even on execution. Only when the file has got MOTW (file downloaded via web browser) it is obligatory checked by the cloud backend via BASF.

The blue arrow shows the moment when the High and High+ (Highest setting in ConfigureDefender) advanced Cloud protection levels are important. They work after uploading the file to the cloud. In this case, the analysis can last longer but we have a lower rate of false positives.

Examples of Metadata used by Defender:

TypeAttribute
Machine attributesOS version
Processor
Security settings
Dynamic and contextual attributesProcess and installation
ProcessName
ParentProcess
TriggeringSignature
TriggeringFile
Download IP and url
HashedFullPath
Vpath
RealPath
Parent/child relationships

Behavioral
Connection IPs
System changes
API calls
Process injection

Locale
Locale setting
Geographical location
Static file attributesPartial and full hashes
ClusterHash
Crc16
Ctph
ExtendedKcrcs
ImpHash
Kcrc3n
Lshash
LsHashs
PartialCrc1
PartialCrc2
PartialCrc3
Sha1
Sha256

File properties
FileName
FileSize

Signer information
AuthentiCodeHash
Issuer
IssuerHash
Publisher
Signer
SignerHash


Edit.
After some additional tests on Windows Server 2019 and Windows 10 Enterprise, I confirmed that files are submitted to the cloud backend also with the "Zero tolerance Block level". The post was edited to reflect this behavior.
 
Last edited:

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176
Cloud Protection Levels

There is a lot of misunderstanding about how Cloud delivered protection works. Here is a diagram from the Microsoft documentation:


View attachment 266439
The red arrow shows the moment when Cloud protection level "Zero tolerance" (Block setting in ConfigureDefender) will block the unknown file. The decision is made in the cloud after the analysis of telemetry (file metadata check). The file is not uploaded to the cloud. Some files can still infect the system if they are not recognized as suspicious by local protection layers - they are not checked by the cloud backend even on execution. Only when the file has got MOTW (file downloaded via web browser) it is obligatory checked by the cloud backend via BASF.

The blue arrow shows the moment when the High and High+ (Highest setting in ConfigureDefender) Cloud protection levels are important. They work after uploading the file to the cloud. In this case, the analysis can last longer but we have a lower rate of false positives.
...

The Microsoft documentation is not perfect and sometimes can be misguiding. Here is some info from the blog of Microsoft MVP:

Cloud-Delivered Protection Blocking Levels​

When the cloud returns a verdict for a submitted file (sample) from the client, you can configure your tolerance-to-risk ratio by configuring blocks level in Microsoft Defender Antivirus:
  • Blocking level = High: applies a strong level of detection while optimizing client performance (greater chance of false positives).
  • Blocking level = High+: applies additional protection measures (may impact client performance and increase risk of false positives).
  • Zero tolerance blocks all unknown executables.

From this info, it follows that "Zero tolerance" Block level can block files after file submission (file is uploaded to the cloud, analyzed, and next blocked). But, there are no additional details available about the difference between Zero tolerance and High+ level. It can be that the only difference is blocking the file instead of sending it to the detonation chamber.(y)
 

Andy Ful

From Hard_Configurator Tools
Thread author
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,176
In my previous post, I suggested that the "Zero tolerance" blocking level can also block files after they have been submitted to the cloud backend. To be sure I tested this behavior on Windows Pro, Windows Server 2019, and Windows 10 Enterprise.
  1. Cloud Protection Level was set to "Zero tolerance" (Block setting in ConfigureDefender).
  2. Defender's Automatic Sample Submission was set to AlwaysPrompt. When this setting is applied, the Windows Security Center claims that "Automatic sample submission is off ", so submissions are not automatical - the user is prompted for consent.
  3. I used KnowBe4 ransomware simulator to run suspicious samples.
  4. A similar test was done with Automatic Sample Submission set to SendSafeSamples (samples without user privacy content are submitted automatically, others are prompted).
With the AlwaysPrompt setting, Defender still prompted sometimes to submit unknown *.cxp files and some AMSI streams used by the simulator to run ransomware.
With the SendSafeSamples setting, Defender showed the alerts that file execution had to be postponed for some seconds (Security scan required) - this alert is related to submitting files to the cloud backend. Also, the submission prompts for AMSI streams were displayed.

So actually, the "Zero tolerance" blocking level does not differ significantly from other blocking levels and the decision that the file is finally unknown & suspicious can be done after the file submission & analysis by the cloud backend.

Here is an updated chart :

1652046689976.png

The blocking levels High, High+, Zero tolerance (or High, Highest, Block in ConfigureDefender) can have an impact on the final "Negative Verdict before defined timeout" marked in red on the above chart.
 
Last edited:

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top