AV-Comparatives Malware Protection Test March 2022

[Anthony Qian]

I totally agree with your opinion. Eset lays too much stress on FP. yes, it's important but not more than protection itself. When I used ESET, I set the protection to all agressive, and it was like other AV's normal protection.

By the way, Do you know how effective Live Guard is? As I know, it's kind of sandbox module just like one in the Avast. I felt the machine learning of ESET(so called Augur) was effective, but I don't know much about Live Guard.

Yes. LiveGuard is a cloud-based sandbox that combines machine learning and behavior analysis engines in multiple layers. Sounds great right? However, in my testing, LiveGuard missed a number of threats which is detected by other trusted vendors and later detected by ESET after manual analysis. After reading the reports of EDTD (=LiveGuard, LiveGuard doesn't provide reports though), I found that those missed samples were marked as "Clean" by the machine learning layer and were not processed by the behavior analysis engines that followed the machine learning engine. Also, the detection threshold of LiveGuard is set to high, possibly due to false positive control for home products.

CyberCapture from Avast isn't a cloud-based sandbox. It's a local dynamic heuristic sandbox that will send suspicious samples to the cloud if the local dynamic engine cannot make a decision.

 

[safetrend]

Thanks for your explanation. It's quite strange that ESET distinguished between EIS and ESS. It's kind of beta feature but ESET promotes it like 'a whole another level of protection.'

 

[devjit2020]

Exactly my grievances with ESET. No doubt their product is the lightest and their signature are one of the best(if not the best) but they simply refuse to listen to customer feedback and Marcos always defends the product when it fails. No product is 100% perfect and the developers should listen to customer feedback. A few of my clients have switched to other products from ESET because they were infected with ransomware. If ESET doesn't have a signature for a ransomware your files will be lost. I don't know how hard it is for the developers of ESET to implement an anti-ransomware module that'll stop the encryption and ask the user for decision when any unknown program atrempts to encrypt their files. Even local antivirus in my country like Quick Heal have this function. ESET is a tier 1 program compared to QH. My ESET license expires this month and I don't think I'll be renewing it. I have no complaints regarding their product but the absolute disregard of the company towards their customers feedback has made me think twice before renewing my subscription.

 

[restwrst]

Would there be any great difference if Eset sensitivity is all set to aggresive?

 

[L0ckJaw]

Would there be any great difference if Eset sensitivity is all set to aggresive?

No difference, just annoying haha

 

[devjit2020]

Would there be any great difference if Eset sensitivity is all set to aggresive?

I haven't seen any difference at least.

 

[amitkumargiri]

Malware Protection Test March 2022

The Malware Protection Test March 2022 assesses program’s ability to protect a system against malicious files before, during or after execution.

www.av-comparatives.org

Well i used to believe these test reports , but nowadays i m doubtful about these...

 

[Anthony Qian]

Would there be any great difference if Eset sensitivity is all set to aggresive?

ML-based detections such as ML/Augur are more likely to be triggered in aggressive mode.

 

[Andy Ful]

That's a whole different test lol, this test you show has the web as the attack vector, while the one we're debating about is the protection against the execution of malware.

Both charts present results when the malware samples are executed.

There are only 4 false positives for Norton but they can impact several tens of thousands of users.
This is more than the summary impact of: Eset + Avira + TotalAV + Kaspersky + McAfee + Microsoft Defender + Malwarebytes + Bitdefender + Total Defenser + Trend Micro + Vipre +Avast

For home users, the strength of Norton comes from Download Insight (reputation lookup) so, Norton will never get a low rate of false positives.

 

[Andy Ful]

...
CyberCapture from Avast isn't a cloud-based sandbox. It's a local dynamic heuristic sandbox that will send suspicious samples to the cloud if the local dynamic engine cannot make a decision.

It is not a local dynamic sandbox. Local heuristics are needed to classify the files (suspicious, non-suspicious). CyberCapture includes a cloud-based sandbox for rare suspicious files. The suspicious file with MOTW is locked for several minutes (up to a few hours). Files without MOTW are ignored by CyberCapture. It works mostly for .exe files - other file types are ignored.

CyberCapture - FAQs | Official Avast Support

A list of frequently asked questions about CyberCapture in Avast Antivirus on Microsoft Windows.

support.avast.com

 

[Arequire]

Well i used to believe these test reports , but nowadays i m doubtful about these...

Why? What changed that you now doubt their validity?

Files without MOTW are ignored.

I've never understood the logic in AV vendors coding their products to perform fewer checks on files not sporting MOTW. It just comes across as vendors deliberately hampering their ability to detect malware pre-execution.

 

[Anthony Qian]

It is not a local dynamic sandbox. Local heuristics are needed to classify the files (suspicious, non-suspicious). CyberCapture includes a cloud-based sandbox for rare suspicious files. The suspicious file with MOTW is locked for several minutes (up to a few hours). Files without MOTW are ignored by CyberCapture. It works mostly for .exe files - other file types are ignored.


View attachment 265956

CyberCapture - FAQs | Official Avast Support

A list of frequently asked questions about CyberCapture in Avast Antivirus on Microsoft Windows.

support.avast.com

What I want to emphasize is that Avast's CyberCapture is not the same as ESET's LiveGuard/EDTD. For most harmful samples, CyberCapture won't send them as a whole to its cloud for analysis; only a few suspicious ones will be sent. As a result, in my testing, many dangerous samples (for example, MBR killer) were rated as "safe to open" by local heuristics, without being sent to the cloud.

LiveGuard is different, everything unknown will be sent to the cloud sandbox for analysis.

In short, the threshold for sending files to the cloud differs, and CyberCapture relies more on its local heuristic engine and LiveGuard relies solely on its cloud sandbox.

I also want to mention another cloud-based detection technology from Avira - Avira Protection Cloud (APC). APC also uses local heuristics when deciding whether or not to send a file to the cloud. It primarily uses cloud-based heuristics to reach a decision in the cloud, allowing it to display results in seconds. It outperforms the above two in terms of detection rate in my testing.

 

[Trooper]

What I want to emphasize is that Avast's CyberCapture is not the same as ESET's LiveGuard/EDTD. For most harmful samples, CyberCapture won't send them as a whole to its cloud for analysis; only a few suspicious ones will be sent. As a result, in my testing, many dangerous samples (for example, MBR killer) were rated as "safe to open" by local heuristics, without being sent to the cloud.

LiveGuard is different, everything unknown will be sent to the cloud sandbox for analysis.

I also want to mention another cloud-based detection technology from Avira - Avira Protection Cloud (APC). APC also uses local heuristics when deciding whether or not to send a file to the cloud. It primarily uses cloud-based heuristics to reach a decision in the cloud, allowing it to display results in seconds. It outperforms the above two in terms of detection rate in my testing.

Good to know about Avira Protection Cloud. I am not really happy with ESET with their cloud analysis at my job. I may have to check their consumer suite out again at some point. It's been years since I have ran it.

 

[Andy Ful]

Why? What changed that you now doubt their validity?


I've never understood the logic in AV vendors coding their products to perform fewer checks on files not sporting MOTW. It just comes across as vendors deliberately hampering their ability to detect malware pre-execution.

The logic is simple. WIndows and AV vendors use MOTW to recognize if the file is downloaded from the Internet and use enhanced protection for such files (SmartScreen, CyberCapture, etc.). Other files are ignored, so the number of false positives is much lower.
If the application makes an update the updater is downloaded without MOTW. If the enhanced security was applied to all files, then many application auto updates could be blocked, especially when enhanced security is based on file reputation (like SmartScreen).

 

[Andy Ful]

...
For most harmful samples, CyberCapture won't send them as a whole to its cloud for analysis; only a few suspicious ones will be sent. As a result, in my testing, many dangerous samples (for example, MBR killer) were rated as "safe to open" by local heuristics, without being sent to the cloud.
...

Interesting observation - worth to be tested.
What happened after running these samples recognized as "safe to open"? Did they manage to infect the system?

 

[Anthony Qian]

Interesting observation - worth to be tested.
What happened after running these samples recognized as "safe to open"? Did they manage to infect the system?

When CyberCapture has marked a file as "safe to open," the file will open automatically. Then, if IDP, Ransomware Protection, AMSI, exploit prevention and other relevant components are able to block this threat, they will. However, none of the above components can identify and block threats like MBR Killer. As a result, the computer was locked......

 

[Andy Ful]

When CyberCapture has marked a file as "safe to open," the file will open automatically. Then, if IDP, Ransomware Protection, AMSI, exploit prevention and other relevant components are able to block this threat, they will. However, none of the above components can identify and block threats like MBR Killer. As a result, the computer was locked......

Are you sure that the sample has got MOTW? The malware samples are usually downloaded in Zip archives and after unpacking, the MOTW can be lost. No MOTW no CyberCapture. Furthermore, CyberCapture can work only for files with the .exe extension.

Edit.
By CyberCapture I mean CyberCapture with sandbox analysis.

 

[Anthony Qian]

Are you sure that the sample has got MOTW? The malware samples are usually downloaded in Zip archives and after unpacking, the MOTW can be lost. No MOTW no CyberCapture. Furthermore, CyberCapture can work only for files with the .exe extension.

Good to know. But in my testing, samples that have been upzipped can trigger CyberCapture. I can see the CyberCapture scanning interface after double-clicking.

Yes. MBR killer is a PE file and has .exe extension.

 

[Andy Ful]

Good to know. But in my testing, samples that have been upzipped can trigger CyberCapture. I can see the CyberCapture scanning interface after double-clicking.

Yes. MBR killer is a PE file and has .exe extension.

What unpacker do you use?

In my tests with Avast, the sample without MOTW was recognized as suspicious and the alert about the scan was visible.

The same sample with MOTW triggered the above alert and additionally the below alert:

 

[Anthony Qian]

What unpacker do you use?

In my tests with Avast, the sample without MOTW was recognized as suspicious and the alert about the scan was visible.

View attachment 265980

The same sample with MOTW triggered the above alert and additionally the below alert:

View attachment 265979

7-zip.