Protomartyr

Level 5
Verified
I'm not sure I agree with Leo's idea of simulating new ransomware by turning off real-time protection (though HIPS is still enabled). It seems like he is associating real-time protection with signatures (starting from 2:48 and onward in the video).

Looking at the ESET documentation, it seems like it's more than just signatures.
Real-time file system protection checks all types of media and is triggered by various system events such as accessing a file. Using ThreatSense technology detection methods (as described in the ThreatSense engine parameter setup section), Real-time file system protection can be configured to treat newly created files differently than existing files. For example, you can configure Real-time file system protection to more closely monitor newly created files.
ThreatSense is technology comprised of many complex threat detection methods. This technology is proactive, which means it also provides protection during the early spread of a new threat. It uses a combination of code analysis, code emulation, generic signatures and virus signatures which work in concert to significantly enhance system security. The scanning engine is capable of controlling several data streams simultaneously, maximizing the efficiency and detection rate.
Sources: Real-time file system protection , ThreatSense engine parameters setup

However, the fact that HIPS failed to block the ransomware is still of concern which Leo did a good job of highlighting in this test.
 

blackice

Level 25
Verified
I'm not sure I agree with Leo's idea of simulating new ransomware by turning off real-time protection (though HIPS is still enabled). It seems like he is associating real-time protection with signatures (starting from 2:48 and onward in the video).

Looking at the ESET documentation, it seems like it's more than just signatures.


Sources: Real-time file system protection , ThreatSense engine parameters setup

However, the fact that HIPS failed to block the ransomware is still of concern which Leo did a good job of highlighting in this test.
It is, and he has his methods which he won’t change. None of these suites are designed to be evaluated with just one of their protections running. It’s a good way to evaluate that one protection, but not the overall efficacy of the product for a home user.
 

SeriousHoax

Level 27
Verified
Malware Tester
I'm not sure I agree with Leo's idea of simulating new ransomware by turning off real-time protection (though HIPS is still enabled). It seems like he is associating real-time protection with signatures (starting from 2:48 and onward in the video).

Looking at the ESET documentation, it seems like it's more than just signatures.


Sources: Real-time file system protection , ThreatSense engine parameters setup

However, the fact that HIPS failed to block the ransomware is still of concern which Leo did a good job of highlighting in this test.
You're right. Real Time Protection does more than just basic signature detection. HIPS can't properly work independently, same goes to the newly implemented AI. They're all interconnected, everything is tied to the Real Time Protection. This a very wrong test. Eset is also known to have no true behavior blocker so I never do any bonus dynamic test in the hub when something is detected by signatures. Testing it without Real Time Protection is kind of stupid. Leo does this always. He grabs a product and test without knowing how the product works.
 

RoboMan

Formerly 'robbie'
Verified
Content Creator
Malware Tester
I hate everything about this test, and I hate Leo's channel. He's creating an army of retarded users, like this one:

1585861205650.png


Throwing thousands of files, using a script to run them all at once, disabling components in order to test others.

Somehow people think they're learning.
 

MacDefender

Level 10
Verified
You're right. Real Time Protection does more than just basic signature detection. HIPS can't properly work independently, same goes to the newly implemented AI. They're all interconnected, everything is tied to the Real Time Protection. This a very wrong test. Eset is also known to have no true behavior blocker so I never do any bonus dynamic test in the hub when something is detected by signatures. Testing it without Real Time Protection is kind of stupid. Leo does this always. He grabs a product and test without knowing how the product works.
When a product like ESET is so dedicated to scan-time analysis (signatures is almost a dismissive way of putting it -- ESET'S runtime protection takes great care to allow their signature scanning to occur at the right times, like when payloads get dropped or decrypted), it's not fair to test it as a pure behavior blocker which it is not.

It's like treating an electric car like a gas car and doing a range test without charging the electric car's battery.

All this test really shows is that ESET doesnt have a behavior blocker, which we understand. Hence, as you said, it doesn’t make sense to do a bonus dynamic test of the product.
But the way Leo is presenting the results is misleading, almost implying ESET is a poor performing product, and that couldn’t be further from the truth. ESET is not doing worse in the Hub compared to other products, even ones with great dynamic behavior blockers.
 
Last edited:

blackice

Level 25
Verified
When a product like ESET is so dedicated to scan-time analysis (signatures is almost a dismissive way of putting it -- ESET'S runtime protection takes great care to allow their signature scanning to occur at the right times, like when payloads get dropped or decrypted), it's not fair to test it as a pure behavior blocker which it is not.

It's like treating an electric car like a gas car and doing a range test without charging the electric saw.

All this test really shows is that ESET doesnt have a behavior blocker, which we understand. Hence, as you said, it doesn’t make sense to do a bonus dynamic test of the product.
But the way Leo is presenting the results is misleading, almost implying ESET is a poor performing product, and that couldn’t be further from the truth. ESET is not doing worse in the Hub compared to other products, even ones with great dynamic behavior blockers.
I always enjoy your voice of reason.
 

SeriousHoax

Level 27
Verified
Malware Tester
When a product like ESET is so dedicated to scan-time analysis (signatures is almost a dismissive way of putting it -- ESET'S runtime protection takes great care to allow their signature scanning to occur at the right times, like when payloads get dropped or decrypted), it's not fair to test it as a pure behavior blocker which it is not.

It's like treating an electric car like a gas car and doing a range test without charging the electric car's battery.

All this test really shows is that ESET doesnt have a behavior blocker, which we understand. Hence, as you said, it doesn’t make sense to do a bonus dynamic test of the product.
But the way Leo is presenting the results is misleading, almost implying ESET is a poor performing product, and that couldn’t be further from the truth. ESET is not doing worse in the Hub compared to other products, even ones with great dynamic behavior blockers.
I always enjoy your voice of reason.
Well said and me too. He doesn't comment much but when he does it's always insightful and educational 👌
 

Cortex

Level 20
Verified
I can't remember beaming Star-Trek style over a 1000 items of malware on to my desktop with my AV either fully or partially disabled - Then running them all at once, that never happens: This has been mentioned many times before malware has to get onto your PC somehow, usually watched by a module of your AV whatever it is - I'm reasonably intelligent (yes i am) but giving me a thousand problems in ten minutes then judging my aptitude by my ability to solve them would not be reasonable (actually thinking on that does seem to happen in this house from time too time)
 

Raiden

Level 17
Verified
Content Creator
However, the fact that HIPS failed to block the ransomware is still of concern which Leo did a good job of highlighting in this test.
Well HIPS in Eset has always been this way. To really get the full capabilities of it, you have to take the time and configure it. If not it's really not going to do too much.

It's why Eset has other modules that work together. No security suite now a days is designed to work this way. They are all designed to PREVENT stuff from getting on to your system in the first place. Malware doesn't just randomly appear, it needs a way in. So testing products like this does absolutely nothing, other than show that this module may not be as strong alone, but together it may be a completely different story.

I dont understand the point of this test at all. For someone who claims to know a lot about this stuff, he surely doesn't show it. Just take the time to explain it and present all of the facts and make it a true learning experience.

I hate everything about this test, and I hate Leo's channel. He's creating an army of retarded users, like this one:

View attachment 236150

Throwing thousands of files, using a script to run them all at once, disabling components in order to test others.

Somehow people think they're learning.
Yup,

I too am not a fan of Leo's tests. I really, really REALLY don't care what the test results are one way or another, I just hate the fact that people are soaking all of this in and assuming everything he is doing is correct. He is misleading people plan and simple and he is making people base their decisions on his videos. Assuming he still works for an AV company, I am still shocked that they let him get away with this stupidity.
 
Last edited:

Durden

Level 3
ESET was my main AV back in the early 2000s (nod32❤).
They were always innovative when it comes to technology. I remember when they released Threatsense engine (I loved the name).

ESET, I haven't used it in years, heck it's been while since I read in detail about the changes it has gone through.
I guess the main reason I abandoned it in the first place was that they didn't jump into the hype of BB's when everybody else seemed to do it, and years past by and they didn't make the move. though they've always had an amazing product, a polished and stable product, and a cutting edge detection.

However, all that changed, ESET implemented a BB module and here in the HUB it shows HOW SOLID the product is. And it's true, ESET is unique, it's more complex than other suites, it doesn't seem that the BB module was designed to function alone, it's not like Emsi's of Deep Guard.
ESET have a multiple modules working together, And even before they implemented the BB module, ESET did detect malware based on behaviour and not signature detection, it's how their realtime protection work, for example, DNA detection (in real time protection) can detect previously unknown malware based on it's malicious behavior ; "DNA Detections can identify specific known malware samples, new variants of a known malware family or even previously unseen or unknown malware which contains genes that indicate malicious behavior."

@darkhobbitmann check this out, it's really brief and fun to read Technology


Advanced users love ESET for a reason, it's not without reason that flawed reviews like this one attract such a strong reaction from MT users who appreciate the value of such product with such great potential. Anyway, in this video Leo seemed more interested in the promotion he talked about.

@SeriousHoax may I ask, I didn't know that ESET finally "unleashed Augur" :D, when did that happen, I mean I remember reading an article about it ! is their and change to the settings regarding it ? and did how much did it influence the product ?
 

MacDefender

Level 10
Verified
When ESET’s signatures work so well that it is regularly outperforming other AVs that use 2-5 engines plus a BB, what is the reason why ESET needs to rush to implement a BB?

A lot of AVs had to resort to using a BB because their signatures performance and response time were not really cutting it with zero-days and obfuscated malware or fileless attacks. ESET's modules are complementary, not overlapping... that means they work together covering different threat vectors. They are not meant to be a bunch of independently acting sub-components that each attempt to provide full protection. More bluntly, in ESET, disabling any protection component severely harms the overall protection of the product. For example, with Emsisoft, running in Bonus BB mode (local signature scanning off) still uses a hash based whitelist+blacklist cloud lookup, basically still providing you signature detections via the Anti-Malware Network. With Norton and F-Secure and friends, disabling network protection is often okay because by the time an exploit payload is downloaded and executing, another component will flag and block it anyway. With ESET, their network protection is a crucial scan point for when new threats get downloaded over the network. Without that, you miss a great opportunity to run the signature scanner.

I think a lot of us agree that if ESET added a dynamic true BB component that performs as well as the top BB's, this would be the be-all-end-all product. But overall, the market benefits from having products approach the problem a different way, IMO more so than everyone trying the same strategy and the only difference being how well each executes it.

TLDR: ESET works differently. If you use it the way it's intended, it performs extremely well. But if you start trying to disable components to test it, disabling the signature scanning component makes the overall product ineffective. This is by design.

P.S. BTW, it's worth mentioning that BB's may be short lived. Behavior Blockers monitor processes by injecting into them and injecting into common attack targets like web browsers or office suites. This process is totally detectable and modern OS process integrity features are starting to clamp down on preventing critical processes from being injected. The drawbacks of a dynamic BB (false positives aside) are that it reduces a monitored process's performance and creates compatibility issues.
 

SeriousHoax

Level 27
Verified
Malware Tester
@SeriousHoax may I ask, I didn't know that ESET finally "unleashed Augur" :D, when did that happen, I mean I remember reading an article about it ! is their and change to the settings regarding it ? and did how much did it influence the product ?
Thanks for your nice constructive reply. Eset implemented Augur into the product around late October last year and recently they also added option to modify detection engine sensitivity from cautious to balanced to aggressive. I've seen some Augur detection but not much yet. Eset preferes avoiding false positives so I guess that's why they haven't unlocked its full potential yet but surely it's going to get better over time.
Here's a screenshot for you: 1.PNG
 

Durden

Level 3
Thanks for your nice constructive reply. Eset implemented Augur into the product around late October last year and recently they also added option to modify detection engine sensitivity from cautious to balanced to aggressive. I've seen some Augur detection but not much yet. Eset preferes avoiding false positives so I guess that's why they haven't unlocked its full potential yet but surely it's going to get better over time.
Here's a screenshot for you: View attachment 236248
Wow that's really something else :D . gotta check it out soon. thanks ^_^
 

Durden

Level 3
Who are you to come and tell me that I eat trash? Do you work at Eset? or are you just a fanboy?
Eset has always had problems detecting unknown malware, with all its modules active or whatever
True, I believe ESET's weak point was indeed unknown malware , and that improved recently. However the main point is the methodology here, we simply pointed out that in order for ESET to detect unknown malware you must not disable real time protection, Real-Time Protection in ESET isn't only signatures and cloud lookup, it also has something to do with behavioral detection.

More importantly, please don't feel personally attacked by anyone here, it's not worth it to get mad at each other for this stuff.☺
 

The Cog in the Machine

Level 21
Verified
The problem is that some people got used to how some products work and handle threats. They just do not get the idea that every vendor has their own ways of dealing with threats. Eset has developed their technology and it has been protecting users very well, right? Eset do not need to develop their own BB simply because vendor X is using one. There are many approaches to prevent infections and the way Eset does that is very impressive.
When I was using Eset on some of may family's devices, I taught them how to check the file reputation using Eset LivGrid; if the file is new and not rated, they'd never run it. What I am trying to say here is you should find yourself a product that works for you and learn how to cover its weaknesses. If Vendor X does not suit you well, go for Y. Eset would have implemented a BB ages ago if they wanted to; they've got the team and the money.
 

Nightwalker

Level 20
Verified
Trusted
Content Creator
ESET has never had a problem with unknown, zero-day malware, it is actually one of their core strengths, anyone who says otherwise doesnt know much about the antivirus industry history and specially how ESET works.

ESET does have a behavior blocker, a sandbox, a emulator, dynamic signatures, advanced machine learning/heuristics and one of the best signatures in the market.

Signatures are commonly seen as old technology, this couldnt be more wrong, signatures are the benchmark of the antivirus industry; great vendors have great signatures (and emulators), they are the results of their malware specialists and R&D work, thats why Kaspersky and ESET are superb products, both are leaders in those two areas.
 

MacDefender

Level 10
Verified
ESET has never had a problem with unknown, zero-day malware, it is actually one of their core strengths, anyone who says otherwise doesnt know much about the antivirus industry history and specially how ESET works.

ESET does have a behavior blocker, a sandbox, a emulator, dynamic signatures, advanced machine learning/heuristics and one of the best signatures in the market.

Signatures are commonly seen as old technology, this couldnt be more wrong, signatures are the benchmark of the antivirus industry; great vendors have great signatures (and emulators), they are the results of their malware specialists and R&D work, thats why Kaspersky and ESET are superb products, both are leaders in those two areas.
Yeah it's worth mentioning that the bulk of the "zero day" malware I've seen around here is generally a variant of something that already exists or a combination of multiple existing malware payloads from the same developers. And ESET's uniquely good at structuring their signatures to detect such zero-day variants compared to any other signature based engine.

With that said, if you truly get hit by zero day malware (never before seen in the wild), ESET will likely miss it. This has been demonstrated in the past as I was writing my own pretend ransomware -- most other behavior blockers reacted to some variants of my test malware but ESET did not at all.

Unless you think you might be targeted by completely custom written malware, I don't consider this to be much of a problem. You are most likely to be hit by emerging variants or repacked versions or obfuscated delivery of already-known malware. Beyond that, as we have demonstrated, there are various ways to completely bypass behavior blockers so the argument of "what if you are being custom targeted?" doesn't really hold water either. A 50-80% detect rate of true zero-days via a behavior blocker is not exactly a reassuring form of protection either when all it takes is one to get through and bring down your infrastructure.
 

Marcos

From ESET
Verified
Developer
I have commented the video as follows:
This test is completely wrong. First of all, you skip the very first layer of defense - Web access protection which is very strong in ESET and blocks download from malicious urls which could save users in many cases from new malware even entering the system. Secondly, by disabling real-time protection you prevent HIPS from receiving events on the file system level and thus make HIPS and all HIPS dependent components ineffective, such as: Ransomware shield, Exploit Blocker, Advanced Memory Scanner, Deep Behavior Inspection, Advanced Machine Learning, etc.

Disabling real-time protection is not just disabling the use of signatures which are, by the way, typically smart DNA signatures in case of ESET, ie. they only describe the malicious behavior to be detected. Disabling RTP prevents other modules from working effectively since they won't receive information about file system events which have nothing to do with signature detection whatsoever.

In real world users must not and do not disable particular protection modules. If they do, they must understand they do it at their own risk and expose the machine to malware attacks and infection.
 
Top