App Review Of LoLBins, 0-Days, ESET, and Microsoft Defender

[cruelsister]

Content source

https://youtu.be/CJBi0EvE398

 

[Bot]

Thanks for sharing this YouTube video link. It's always beneficial to have visual content that can further explain and give insights about LoLBins, 0-Days, ESET, and Microsoft Defender. Let's discuss the points raised in this video.

 

[SeriousHoax]

Usually, I see Microsoft Defender allowing Lobbins to connect and download files but nice to see that in this case it reacted promptly. But the main question is, was the downloaded file a true malware or is it a benign file and you just wanted to check if the mechanism of Lolbins connecting and downloading is blocked by MD & ESET?

 

[Practical Response]

Usually, I see Microsoft Defender allowing Lobbins to connect and download files but nice to see that in this case it reacted promptly. But the main question is, was the downloaded file a true malware or is it a benign file and you just wanted to check if the mechanism of Lolbins connecting and downloading is blocked by MD & ESET?

This is a good question based on the "generic detection" for suspicious behaviors that can detect "potentially malicious" files demonstrated by Microsoft.

Also as you stated MS has been known to miss detection's on these as well as none are 100% detection, especially when modifications of files until they bypass are presented in this form as well as directly from the desktop bypassing route of infection.

 

[cruelsister]

Usually, I see Microsoft Defender allowing Lobbins to connect and download files but nice to see that in this case it reacted promptly. But the main question is, was the downloaded file a true malware or is it a benign file and you just wanted to check if the mechanism of Lolbins connecting and downloading is blocked by MD & ESET?

Yes, in this case the mechanism was tested. The test files had the ability to connect to an external server and successfully download a file which could be benign or malicious- without a peep from E, but blocked by D.

Also as you stated MS has been known to miss detection's

Being a Kind and Gentle person, it felt it appropriate to give Defender a little love (Ophelia hissed at me for this).

 

[Practical Response]

Yes, in this case the mechanism was tested. The test files had the ability to connect to an external server and successfully download a file which could be benign or malicious- without a peep from E, but blocked by D.

Being a Kind and Gentle person, it felt it appropriate to give Defender a little love (Ophelia hissed at me for this).

So basically your faulting Eset for not stopping what is most likely not malicious and giving defender credit for flagging something most likely not malicious. Am I following your logic correctly here?

Don't get me wrong, I love to see days when defender holds its own, but the method and legitimacy is important.

 

[SeriousHoax]

Yes, in this case the mechanism was tested. The test files had the ability to connect to an external server and successfully download a file which could be benign or malicious- without a peep from E, but blocked by D.

I think there are both upsides and downsides to this. To give an example of my own, I use an app and the easy way to download it is to paste the code on Terminal and it will use PowerShell to download/update the app and install it. With AVs like Kaspersky and Bitdefender, I have to turn off their protection to download this as they block the mechanism while with default MD and ESET I don't have to as MD allows downloading via powershell.exe and ESET doesn't usually detect malware unless their protection layers find suspicious/malicious code in them or web protection blocking blacklisted sites.
So, though it's a good idea, there could be some cases where it would create false positives and annoyance.

 

[LennyFox]

Yes, in this case the mechanism was tested. The test files had the ability to connect to an external server and successfully download a file which could be benign or malicious

This behavior by itself is suspicious enough IMO to block it (assuming it was not signed by a trusted vendor)

 

[cruelsister]

your faulting Eset for not stopping what is most likely not malicious

Actuslly just demonstrating how some products deal with common mechanism utilized by a typical trojan downloader. Such things can be coded to connect to legitimate websites and download legitimate things, whereas it could also connect out to the Darkness Beyond and download things that will yield No Joy to the user.

No bashing of E was done or intended, just information for those that use and like ESET.

but the method and legitimacy is important.

Agreed. Running an unknown file to view its effects on a product protected system is indeed important, and that was what was done.

 

[Practical Response]

Yes, in this case the mechanism was tested. The test files had the ability to connect to an external server and successfully download a file which could be benign or malicious- without a peep from E, but blocked by D.

Did the mechanism happen to be the lolbin "BITS" which is used to download Windows updates ?

This would not only cause the AV to see this as a user initiated request but the AV in case "eset" uses emulation that will not detect an already hard to detect item if it contains no malicious code.

The detection of MS tells me this.

If this is indeed the case it's nothing more than slight of hand as it could be considered a fail, but really its not on Esets part.

 

[Andy Ful]

The detection of Microsoft Defender is acceptable, because the Certutil LOLBin was used in the attacks many times and people rarely use it to download files. However, the detection could be improved by allowing files with good reputations. In the case of the test, the file should be allowed (has a good reputation), but the file with an unknown reputation might be blocked.

The detection of Eset in the test is also acceptable because the downloaded file has a good reputation.
I think that @cruelsister's test can be extended as follows:

1. Download the file that SmartScreen blocks in Edge (SmartScreen set to "Block potentially unwanted apps") by choosing "keep the file".
2. Check if the file from point 1 is undetected by Eset.
3. If points 1 and 2 are fulfilled, download the file again using the Certutil LOLBin.

The extended test is based on the rational idea that potentially unwanted apps downloaded via suspicious methods should be prevented. It is possible, that Eset will fail such a test.
The @cruelsister's test shows that Microsoft Defender detection can be improved. The extended test (if Eset fails) can show that Eset's detection can be improved.

 

[Victor M]

Is there a way to configure ESET to detect such LoL bin usage ?

 

[Practical Response]

The detection of Microsoft Defender is acceptable, because the Certutil LOLBin was used in the attacks many times and people rarely use it to download files. However, the detection could be improved by allowing files with good reputations. In the case of the test, the file should be allowed (has a good reputation), but the file with an unknown reputation might be blocked.

The detection of Eset in the test is also acceptable because the downloaded file has a good reputation.
I think that @cruelsister's test can be extended as follows:

1. Download the file that SmartScreen blocks in Edge (SmartScreen set to "Block potentially unwanted apps") by choosing "keep the file".
2. Check if the file from point 1 is undetected by Eset.
3. If points 1 and 2 are fulfilled, download the file again using the Certutil LOLBin.

The extended test is based on the rational idea that potentially unwanted apps downloaded via suspicious methods should be prevented. It is possible, that Eset will fail such a test.
The @cruelsister's test shows that Microsoft Defender detection can be improved. The extended test (if Eset fails) can show that Eset's detection can be improved.

If that's the case certutil is used by admin and developers and can be used to download files, it's part of the certificates service and it's used to display, configure and restore CA components.

Again if the file contained no actual malicious code ,what in it is to trigger Esets scanning, the script actually appearing to look as if the user initiated it.

These POCs are getting to be very misleading.
Executed from the desktop environment, calls out to drop a benign file. MS detects it with a generic aggressive signature not because it's malicious but "has possibility"

Put an actual payload in it and run it through again. This time use real world route of infection as well.

 

[Andy Ful]

If that's the case certutil is used by admin and developers and can be used to download files, it's part of the certificates service and it's used to display, configure and restore CA components.

It is not a common way. Admins usually download files by using PowerShell.

Put an actual payload in it and run it through again. This time use real world route of infection as well.

Such a test might hardly be done in practice. One should find many malicious payloads undetected by Eset and next download them via the Certutil to see how many files could be blocked. It is easier to do it with non-malicious files with unknown reputations.

The test with non-malicious files (proposed by me) can show if the AV uses aggressive behavior blocking or not. The people (including admins) can choose which AV is better for them

 

[Practical Response]

It is not a common way. Admins usually download files by using PowerShell.

I find you apply enterprise to a home users based forum often.

Such a test might hardly be done in practice. One should find many malicious payloads undetected by Eset and next download them via the Certutil to see how many files could be blocked. It is easier to do it with non-malicious files with unknown reputations.

The test with non-malicious files (proposed by me) can show if the AV uses aggressive behavior blocking or not. The people (including admins) can choose which AV is better for them

If the file does not contain malicious code, it can not be detected by web filter protection, or upon post execution on the system by heuristics and behavioral not to mention HIPS based protection layers that monitor behaviors. This leaves the exercise inaccurate and only reflects on the abilities of products to be so aggressive they produce false positives for non malicious items using generic detection's.

How does one go about choosing a product properly with these misgivings?

Again so we are clear, I'm not a product fanboy, I don't even use any of these products, I am however a fan of doing things fairly, and not misleading when it comes to helping users secure their homes and environments. If one has issue with a products country or maker, they should just state it, or if they prefer one solution to the next again, just be open about it, but these methods are misleading at best because it does not test the products as designed, it just demonstrates concepts that are not realistic.

 

[Andy Ful]

I find you apply enterprise to a home users based forum often.

Home users do not use Certutil (and most of LOLBins) at all. It is a tool used mostly by developers and IT administrators. The Microsoft Defender detection of Certutil will be the same for home users and organizations.

If the file does not contain malicious code, it can not be detected by web filter protection, or upon post execution on the system by heuristics and behavioral not to mention HIPS based protection layers that monitor behaviors.

This is not true. In many cases, the legal Administrative tools are detected as malware. For example, many of my applications are initially detected as malicious. The detection can survive for a long time until the developer reports it as a false positive (sometimes this will not help too). Also, SmartScreen in Edge can block many legal applications if they do not have a good reputation.

Another example can be the latest AVLab test, where most AVs detected two legal remote admin applications as malware. Microsoft left those tools undetected. Many PUAs can be detected as malicious, especially when they are abused as a part of an attack.
It can happen that the attack will be stopped by detecting/blocking legal PUA or by blocking the LOLBin which wants to download PUA. Such differences can be interesting for some people.

Anyway, you are right in saying that tests with POCs cannot say much about the overall AV protection. Such tests are often misinterpreted by people.
Are tests with POCs informative? Yes, some of them can show interesting differences between the AV detections.

 

[Practical Response]

Home users do not use Certutil (and most of LOLBins) at all. It is a tool used mostly by developers and IT administrators. The Microsoft Defender detection of Certutil will be the same for home users and organizations.



This is not true. In many cases, the legal Administrative tools are detected as malware. For example, many installers of my applications are initially detected as malicious. The detection can survive for a long time until the developer reports it as a false positive (sometimes this will not help too). Also, SmartScreen in Edge can block many legal applications if they do not have a good reputation.

Another example can be the latest AVLab test, where most AVs detected two legal remote admin applications as malware. Microsoft left those tools undetected. Many PUAs can be detected as malicious, especially when they are abused as a part of an attack.
It can happen that the attack will be stopped by detecting/blocking legal PUA or by blocking the LOLBin which wants to download PUA. Such differences can be interesting for some people.

Anyway, you are right in saying that tests with POCs cannot say much about the overall AV protection. Such tests are often misinterpreted by people.
Are tests with POCs informative? Yes, some of them can show interesting differences between the AV detections.

Wanting to see if a product is triggered by actions that mimic behaviors leading to what you have described is a false positive, it is certainly not the same as a benign file with no payload waltzing out and back into the front door initiated by the user. I was not exact in my wording and it appears loosely translated although my meaning I know you understand, as the file in this case is not performing a malicious action nor contains malicious code. It is being used to demonstrate a way that file could be used to carry that out, but not actually doing so.

Thank you though for placing what is necessary here in a extra disclaimer that users understand the product is not failing because its not actually being tested by its design and abilities. It is literally a concept. Personally I would not want to use a security system that is so aggressive. Its like having a dog that barks at everything around your house, how would you know what is a legitimate threat or not, especially for those average users.

 

[Andy Ful]

Personally I would not want to use a security system that is so aggressive. Its like having a dog that barks at everything around your house, how would you know what is a legitimate threat or not, especially for those average users.

Many AVs have such aggressive behaviors. Some can block legal tools, others can block LOLBins.
Some time ago I tested an AV that blocked by default the outbound Internet connections of PowerShell.
When I tested ZoneAlarm in the Challenge series, it blocked the shortcut with benign CmdLines containing cmd[.]exe.

Eset is known to have almost no false positives (on default settings), but its detection is not among top AVs. Of course, Eset has got HIPS that can strengthen the protection with a cost of false positives.

Edit.
Microsoft uses Machine Learning to decrease the number of false positives. The ML model probably recognized that using Certutil to download files is very uncommon among customers, so such behavior can be detected as malicious.

 

[Practical Response]

Many AVs have such aggressive behaviors. Some can block legal tools, others can block LOLBins.
Some time ago I tested an AV that blocked by default the outbound Internet connections of PowerShell.
When I tested ZoneAlarm in the Challenge series, it blocked the shortcut with benign CmdLines containing cmd[.]exe.

Eset is known to have almost no false positives (on default settings), but its detection is not among top AVs. Of course, Eset has got HIPS that can strengthen the protection with a cost of false positives.

Edit.
Microsoft uses Machine Learning to decrease the number of false positives. The ML model probably recognized that using Certutil to download files is very uncommon among customers, so such behavior can be detected as malicious.

Eset much like CIS is not designed to be run out of the box "hence the elaborate settings" even though it has default settings for average users. Those are a mix of security and usability. Demonstrating if a product misses something legitimate, without demonstrating its capabilities, is pointless. Being advanced enough to use the product to its fullest capabilities is another matter altogether which is much like not testing it that way.

How high of a detection rate does one product have to have to become usable for average users, i have read it can be as high as 99.8 percent of almost all wide spread malware. I see users here that agree that they have not had an infection in ages just using default securities, liking these posts. Im unclear as to why when these products are being tested in a manor that resembles parlor tricks.

Microsoft is fully aware of its issues and LOLBins and their vulnerabilities. Lists for these have been around since the early 2000's and used by a lot of products including yours. Microsoft itself has pointed out and is also something understood in the enterprise arena that with LOLBins the best solution is , if its not being used, disable it.

That said MS Defender is also known to have FPs as well from such detection's as you described.

Bottom line here is not that the file could be used for something, it was not fitted with a malicious payload nor did it actually commit any malicious actions other then its not common to be used, but again, the detection by microsoft was a generic response to this "possibility" , so props up to it flagging a benign file, but it is not an indicator the other product failed. Nor would I suggest users remove this product because they watched this test and think it failed.

 

[Andy Ful]

Bottom line here is not that the file could be used for something, it was not fitted with a malicious payload nor did it actually commit any malicious actions other then its not common to be used, but again, the detection by microsoft was a generic response to this "possibility" , so props up to it flagging a benign file, but it is not an indicator the other product failed.

The AV vendors already decided that they should detect some legal applications or restrict some LOLBins, just because they could be possibly used for something malicious. If you ask them, you can get the answer that it is necessary to provide sufficient protection. The fact that You or I might not like the answer will not change anything. Such practice is forced by the attackers who use the attack methods that abuse legal resources and AV vendors cannot find a better way.

Nor would I suggest users remove this product because they watched this test and think it failed.

Agreed.
The last sentence in the video can be questionable without additional comment, but I think that the author of this thread made it clear in her last post: "No bashing of E was done or intended, just information for those that use and like ESET." I hope that the (slightly too long) discussion in this thread helped readers to avoid misunderstanding the test results.