App Review Of LoLBins, 0 Days, and ESET (Part 2)

[Trident]

Microsoft can also use sandboxing, but it works differently. The file execution is suspended for a short time (10-60 seconds) and some malware can infect the system before the analysis in the sandbox is finished. So, blocking the popular methods can be reasonable for Microsoft, but not necessary for Eset Protect Advanced.

Once again, we are reaching the point where it should be mentioned that complete copy and paste of methods from vendor to vendor is not necessary.

For the highest efficiency against fileless malware, it is of extreme importance that generic methods are developed — e.g evaluating LOLBins command lines and terminating execution there and then, as well as working closely with the AMSI and proprietary hooking.
Just relying on web blocking and definitions is not enough as website/domain name may be new (or generated for the attack at the spot) as well as definitions on obfuscated code have 0 effectiveness.

Having cloud emulation (sandbox) is another great approach.

Then the question comes how important is blocking fileless attacks on home users machines.
We do not have enough information to conclude how prevalent these are.

 

[Practical Response]

Passive heuristics or behavior based heuristics may of detected the fileless but who is to say when not given the chance. That latter part the most important, when not given the chance but judged upon.

The question of how important these are of blocking on home users machines and how prevalent they are is another good question.

Again it is not just about Eset, but testing in general I am pointing out.

 

[Shadowra]

As for @Shadowra this comes with no offense meant but because he threw you out there like this I have no choice but to comment.

This user tests URLS and files from a folder on a desktop, not true route of infection testing just like has been done here.

And how would you like me to do that?
I don't have a way of injecting the samples via the network or anything :/.

 

[Practical Response]

And how would you like me to do that?
I don't have a way of injecting the samples via the network or anything :/.

Downloading files from a file sharing site is the same method used as any other site, already mentioned this, and did ask for you to not take offense, but I will not be silenced because Lenny threw you out in front of himself.

 

[czesetfan]

At one time, BD used to offer something along that line, regarding the below. I don't remember what the the options for the Firewall were.

View attachment 282951

Yes, I was thinking exactly something like that, but for the overall product setup.

Everyone can be customized somehow, but it's a lot of small adjustments in a branching tree of advanced settings. I'm imagining something like driving modes on modern cars. One click - Normal, Sport, Eco - changes the characteristics of the whole car at once: even the engine, gearshift, chassis and so on.
Something along these lines AV products tend to have in the form of a "Quiet, Game" mode, where the AV's characteristics are adjusted so as not to disturb the game for example. Thus not triggering scans, not displaying prompts, and the like.
So, for example, a "Harder" mode would tighten the AV engine's detection threshold, add some tightening rules for HIPS, firewall, web access detection, etc.

I think it's all about psychology and economics, not about technicalities, software coding, or malware research. I suppose there is currently only a two-tier customer split with AV developers. The "basic - mainstream" user, to whom features and settings are tuned "out of the box". And then there are the complicated settings for more experienced users, who are expected to understand them and be able to tweak their system individually by changing them.

Basic settings, e.g. at ESET, are tuned in the mentioned triangle - detection, performance, friendliness. That is, none of them is emphasized, so it is not tuned to the maximum possible. Better detection potentially increases the load and threatens FP, necessitating user interaction - we don't want that. Why? Because statistically it is worse to have 10-100 FP cases, a complaint about higher PC load, queries to Manufacturer Support, than to miss a sample 1 time. This is treated by the formulation - no AV provides 100% protection. Which is also the objective truth in essence. That's why in various standard tests, good AVs oscillate around 98-99.9% success rate. Conversely, with specific ones, the differences can be much greater, which I think is the case here.

Is it technically possible to treat this kind of attack? Certainly yes. The ESET team can do it Of LoLBins, 0-Days, ESET. But the question is what it will cost in effort, time and money against the real probability of such an attack in a "Basic" user. As Marcos mentioned in the ESET forum. Blocking a legitimate app just because it can download malware is not ideal on a global scale. (But could it be in "Hard" mode?) It could be refined with HIPS, firewall rules, but that's just it. Someone has to troubleshoot it, program it, debug it, etc.
That's possible in a corporate environment where they pay an external team or their own security department to do it, but does it make sense for a "Basic" user's setup in millions of installations?

And there are many such decisions, from day-to-day operations to the strategic direction of the entire product. What is effective, what to develop, what will it deliver and at what cost?

"Basic" user installs and doesn't address. He doesn't want to and doesn't know how to deal with potential issues and complications. (After all, ESET doesn't even have PUA detection turned on in the basic setup, and yet probably a pretty high percentage of detections fall into this category.) The "advanced" user has plenty of options to "play" with the settings. But as I say, I'd like the ability to incrementally, "user frendly" harden/improve basic protection in simple steps, without detailed knowledge of the engine. Come to think of it, isn't splitting products and adding layers of protection (NOD32, Internet Security, Premium Security, Ultimate) just an example of this approach? Of course there is marketing behind it .

So can it be done better? Technically, it certainly does. The question is just the right "tuning" of detection/protection, performance, friendliness, development team time, cost, etc. And that's probably why there are differences between AV vendors that we perceive in tests and the reality of the "world out there".

But maybe it's different, I'm not saying I'm right.

Ohh, long enough text..

 

[Trident]

And there are many such decisions, from day-to-day operations to the strategic direction of the entire product. What is effective, what to develop, what will it deliver and at what cost?

Every vendor has different vision how their product should work and protect users, although to a large extent modules, features and functions overlap.
Some focus on reputation, others on machine learning, Eset is heavily focused on accurate, most of the time generic detections, that are machine-generated (documentation available as well) and on their local emulator for scripts, and executables.
It is possible to activate more aggressive modes for the local emulator and a cloud emulator, including what they call “sandbox on steroids” (experimental) is available as well for downloads.

It is possible to do a lot more to block fileless attacks, for example, I am fairly certain that no admin will start executing complicated scripts nowadays. And even if they do, I am certain they won’t be obfuscated and won’t include commands such as “join”, “split” and many others. I am also confident no admin will attempt to download files via certutil.

However, Eset’s Research and Development team most likely hasn’t found the necessary evidence that their current approach doesn’t work and have hence, not implemented more generic blocks.

Detection (signature) -> generic detection -> heuristic (short and effective logic) -> behavioural profile -> TTP-based behavioural analysis -> policy-based behavioural blocking (lockdown mode)…
The above progresses to more and more generic methods that can block more attacks and can also, create more false positives.
Home products are more or less “one size for all” solutions.
On business environments there are MSSPs that will analyse the solution weak points as well as the environment and will implement additional security measures.

 

[devjit2020]

Eset is very bad, I never felt protected when I used it.
Greetings.

For the average home user ESET is perfect. In my 5 years of using it none of my PC’s have been infected. It runs on my gaming PC without any problems which is unfortunately not true with Kaspersky & Bitdefender in my case. So ESET is not bad. I have been infected many times using Bitdefender. So will I say that it is bad? Generalising your opinion on a certain product seems to have become a rule on MT. Only a handful of members discuss about real topics. ESET may be bad for you but in general it is a top tier AV. No matter how much you scream the fact won’t change. Just because you are not happy with a product does not mean that in general the product is bad. In that case I can also claim that BD and KIS are thrash. But they are not. Look at the various testing websites. All of them are top tier Av’s. ESET FW & HIPS can be tweaked for maximum protection. Most people don’t know that. If you want maximum protection configure some rules in the firewall & hips and then run the test again to see the results.

 

[devjit2020]

I'm going to post a probably not popular view but none the less the double standards also set with not testing as designed as I already mentioned.

If CS were to test CIS against this, all of you would expect to see it done with CS's set up she uses, basically tweaking the settings. This is normal because CIS was designed to be tweaked. Yet the testing of this product also designed to be tweaked is tested at defaults and no one ever questions that. Eset is designed to be hardened by those that know how and are capable.

So again as stated, this is certainly not real world testing.

I will partially agree with this on one point. CIS is always tweaked when running tests but at the same time CIS is based on proactive protection and so it will fare better against unknown malware as compared to ESET. But yes for keeping a fair comparison if CIS is always tweaked for the tests, others should also be tweaked and not kept at the default settings.

 

[cruelsister]

Food for thought: When you know how the chain of attack is executed and you create HIPS rules to stop the attack, than it would hardly qualify as a zero day would it?

A Most Excellent point! Although one could very well set up HIPS rules to detect/prevent a given type of malware, such a Rule could be considered as reactive to something specific and not at all proactive in general (sorry if I didn't write that clearly).

But making Rules reactively for a product implies covering up deficiencies in that product and leads to sort of a Whack A Mole security philosophy which personally I am not in favor of.

 

[Practical Response]

A Most Excellent point! Although one could very well set up HIPS rules to detect/prevent a given type of malware, such a Rule could be considered as reactive to something specific and not at all proactive in general (sorry if I didn't write that clearly).

But making Rules reactively for a product implies covering up deficiencies in that product and leads to sort of a Whack A Mole security philosophy which personally I am not in favor of.

Security deficiencies. Interesting topic. Since around 2003 when the government first listed about lolbins vulnerability it's been well known and established. MS has made it well known since then if it is not needed it should be disabled. Windows obviously was never meant to be a home user product as it's evident when you realize none of these users set an admin account and place standard ones on as designed, it's security and process too hard for most to figure out.

Both MS and products will protect users from themselves to a point and then hide behind the EULA beyond that point, placing blame on the end user.

Many vendors have their hands full because average users can barely install products like Eset, let alone know their infected, but they will certainly figure out how to disable a product to install something they want on their system bad enough. So does it make sense to harden the product out of the box or to place advanced settings for those that know and can.

So let's be honest, who is covering up deficiencies and who is addressing them in the only manor effectively available.

 

[Trident]

A Most Excellent point! Although one could very well set up HIPS rules to detect/prevent a given type of malware, such a Rule could be considered as reactive to something specific and not at all proactive in general (sorry if I didn't write that clearly).

But making Rules reactively for a product implies covering up deficiencies in that product and leads to sort of a Whack A Mole security philosophy which personally I am not in favor of.

You are right here, I have to agree.
I also have to add that Eset HIPS capabilities are far from broad and whilst yes, additional protection could be implemented, this protection would hardly be sufficient. A lot will remain uncovered.

However, this all applies to scripts (in HIPS powershell, CMD, consolehost, wscript and cscript could be prevented from launching processes and in firewall they can be disconnected), maldocs (rule to prevent MS office from launching child processes could be created in Eset HIPS and it will cover 100% of this vector), Java malware (javaw.exe could be blocked in firewall) and others.
We don’t know how many home users come across such malware.

This test (much like one that I did about a year ago) shouldn’t be used as an indicative of a product’s effectiveness, it should be observed out of curiosity.

It would’ve been better if you downloaded the file (for example from OneDrive) so we can see LiveGuard emulation.

Yes, malicious file can be on external storage, but:

• To go on external storage, it will most probably be downloaded or saved through emails and may be emulated by Eset (or another product).
• It may be a worm such as Dinihou, but you will not operate with external storage every hour, so the threat will be covered by signatures by the time you plug external media in.
• It is possible for admins under certain conditions to disable external media usage.

So for me, it would be better to use downloads and ensure LiveGuard (which is so-so on executables) is in use — it is there for a reason. If in this case the product still fails, shame on them.

 

[LennyFox]

I will partially agree with this on one point. CIS is always tweaked when running tests but at the same time CIS is based on proactive protection and so it will fare better against unknown malware as compared to ESET. But yes for keeping a fair comparison if CIS is always tweaked for the tests, others should also be tweaked and not kept at the default settings.

Although a valid point in general, two remarks here:

1. Cruel Sister was the first to publish these tweaks (hence they are known as CS settings). So you can't blame her that she does test with Comodo using her non default settings. Her settings by the way have become the default settings for Comodo's business version (Xcitium), so what is considered default is not as black and white as stated anymore (CS rules are non default for Comodo, but are the default for Xcitium and Xcitium business version is not free, so free CIS being used with CS settings is to some degree justifiable).
   
   
2. In the first video an untweaked Defender was compared to an untweaked ESET, so that was an equal terms comparison (both on default). Microsoft Defender can also be hardened using Andy's configure Defender or Dan's Defender UI.

 

[Trident]

Although a valid point in general, two remarks here:

1. Cruel Sister was the first to publish these tweaks (hence they are known as CS rules). So you can't blame her that she does test with Comodo using her non default settings. Her settings by the way have become the default settings for Comodo's business version (Xcitium), so what is considered default is not as black and white as stated anymore (CS rules are non default for Comodo, but are the default for Xcitium and Xcitium business version is not free, so free CIS being used with CS rules is to some degree justifiable).
   
   
2. In the first video an untweaked Defender was compared to an untweaked ESET, so that was an equal terms comparison (both on default) Microsoft Defender can also be hardened using Andy's configure Defender or Dan's Defender UI)

Everyone can test the way they want to test. If someone wants to test a product tweaked, then so be it. There are no rules set in stone how testers should act.
I just have the following recommendations:

• Always make sure that all protection components are tested and not just one. If a component is not tested, then explain why, for example “I am not testing LiveGuard as it covers files up to X MB and I can easily fill a string with garbage to exceed this size”.
• When using variety of malware that may evade detection, make it clear to what extent this malware affects home users.
• When performing tests with a single sample, always aim to provide recommendations on how users can protect their systems so something useful can be drawn out of the test and it doesn’t look like plain product bashing/promoting.

 

[LennyFox]

Everyone can test the way they want to test. If someone wants to test a product tweaked, then so be it. There are no rules set in stone how testers should act.

Well professional testing organizations sort of apply a gentlemen's agreement that they do comparative test in default settings.

I agree with you that It would provide better insights when testers would explain which components were not tested or tested specifically (that is what I like about the videos of CS, she always clearly describes what she tests).

 

[Practical Response]

Well professional testing organizations sort of apply a gentlemen's agreement that they do comparative test in default settings.

They also agree real world protection testing methods are standard as well which very much includes route of infection.

 

[Trident]

Well professional testing organizations sort of apply a gentlemen's agreement that they do comparative test in default settings.

I agree with you that It would provide better insights when testers would explain which components were not tested or tested specifically.

The disclaimer on top could be updated to better reflect that test, although credible, may not necessarily reflect the product’s performance in real-life situations. @Jack

Current disclaimer mentions briefly that such tests should be taken with a grain of salt but does not explain why.

 

[devjit2020]

I’m not into any argument there. We all know the efforts Shadowra & CS puts into their videos. My point is that these two tests do not prove that ESET is a weak product as one member states. Defender & Eset handles threats in different ways. We all know Eset has one of the best signatures and hence it’s mostly dependant on these. WD has a mixture of ML, cloud & sigs. If you look at another angle, you’ll see WD CF access blocking many legitimate apps and in the false positive tests it also shows. Eset has no FP. Their goal is to set a balance between detection & usability. An advanced user can tweak the settings to make it a lot more stronger at the cost of usability. I was once a tester of ESET in this forum (username wraith ). With my configured settings, added FW & HIPS rules, I’ve yet to become infected (except once because of my own foolishness).

 

[Practical Response]

An advanced user can tweak the settings to make it a lot more stronger at the cost of usability. I was once a tester of ESET in this forum (username wraith ). With my configured settings, added FW & HIPS rules, I’ve yet to become infected (except once because of my own foolishness).

Most of the users in this thread do not understand the products capabilities and how to add rules to the HIPS/Firewall and the extent at which one can customize let alone how the operating system they would need to know to do so works. Yet they claim to be experts in the applications abilities. They will slam a user for pointing out issues with a test according to design of a product, without fully understanding what they are even viewing. The issue with the testing was not whether the product passed or failed but how it was tested in order to give it a fair shake as per design even at default settings. Either way, judging the product based off the tests and personal opinions of those that have no clue how to properly use the product would be foolish at best.

 

[czesetfan]

Doesn't this current testing just point to the questions in this topic?
Protection Effectiveness Of EDR Solutions Against Internet Threats » AVLab Cybersecurity Foundation