AV-Comparatives Malware Protection Test March 2022

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Anthony Qian

Level 10
Verified
Well-known
Apr 17, 2021
453
I totally agree with your opinion. Eset lays too much stress on FP. yes, it's important but not more than protection itself. When I used ESET, I set the protection to all agressive, and it was like other AV's normal protection.

By the way, Do you know how effective Live Guard is? As I know, it's kind of sandbox module just like one in the Avast. I felt the machine learning of ESET(so called Augur) was effective, but I don't know much about Live Guard.
Yes. LiveGuard is a cloud-based sandbox that combines machine learning and behavior analysis engines in multiple layers. Sounds great right? However, in my testing, LiveGuard missed a number of threats which is detected by other trusted vendors and later detected by ESET after manual analysis. After reading the reports of EDTD (=LiveGuard, LiveGuard doesn't provide reports though), I found that those missed samples were marked as "Clean" by the machine learning layer and were not processed by the behavior analysis engines that followed the machine learning engine. Also, the detection threshold of LiveGuard is set to high, possibly due to false positive control for home products.

CyberCapture from Avast isn't a cloud-based sandbox. It's a local dynamic heuristic sandbox that will send suspicious samples to the cloud if the local dynamic engine cannot make a decision.
 
Last edited:

devjit2020

Level 2
Apr 7, 2022
91
Exactly my grievances with ESET. No doubt their product is the lightest and their signature are one of the best(if not the best) but they simply refuse to listen to customer feedback and Marcos always defends the product when it fails. No product is 100% perfect and the developers should listen to customer feedback. A few of my clients have switched to other products from ESET because they were infected with ransomware. If ESET doesn't have a signature for a ransomware your files will be lost. I don't know how hard it is for the developers of ESET to implement an anti-ransomware module that'll stop the encryption and ask the user for decision when any unknown program atrempts to encrypt their files. Even local antivirus in my country like Quick Heal have this function. ESET is a tier 1 program compared to QH. My ESET license expires this month and I don't think I'll be renewing it. I have no complaints regarding their product but the absolute disregard of the company towards their customers feedback has made me think twice before renewing my subscription.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,458
That's a whole different test lol, this test you show has the web as the attack vector, while the one we're debating about is the protection against the execution of malware.

Both charts present results when the malware samples are executed.


1650223433877.png


1650223484928.png


There are only 4 false positives for Norton but they can impact several tens of thousands of users.
This is more than the summary impact of: Eset + Avira + TotalAV + Kaspersky + McAfee + Microsoft Defender + Malwarebytes + Bitdefender + Total Defenser + Trend Micro + Vipre +Avast

For home users, the strength of Norton comes from Download Insight (reputation lookup) so, Norton will never get a low rate of false positives.
 
Last edited:

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,458
...
CyberCapture from Avast isn't a cloud-based sandbox. It's a local dynamic heuristic sandbox that will send suspicious samples to the cloud if the local dynamic engine cannot make a decision.

It is not a local dynamic sandbox. Local heuristics are needed to classify the files (suspicious, non-suspicious). CyberCapture includes a cloud-based sandbox for rare suspicious files. The suspicious file with MOTW is locked for several minutes (up to a few hours). Files without MOTW are ignored by CyberCapture. It works mostly for .exe files - other file types are ignored.


1650226471364.png


 
Last edited:

Arequire

Level 29
Verified
Top Poster
Content Creator
Feb 10, 2017
1,822
Well i used to believe these test reports , but nowadays i m doubtful about these...🤔🤔🤔🤔🤔🤔🤔🤔🤔🤔
Why? What changed that you now doubt their validity?

Files without MOTW are ignored.
I've never understood the logic in AV vendors coding their products to perform fewer checks on files not sporting MOTW. It just comes across as vendors deliberately hampering their ability to detect malware pre-execution.
 

Anthony Qian

Level 10
Verified
Well-known
Apr 17, 2021
453
It is not a local dynamic sandbox. Local heuristics are needed to classify the files (suspicious, non-suspicious). CyberCapture includes a cloud-based sandbox for rare suspicious files. The suspicious file with MOTW is locked for several minutes (up to a few hours). Files without MOTW are ignored by CyberCapture. It works mostly for .exe files - other file types are ignored.


View attachment 265956

What I want to emphasize is that Avast's CyberCapture is not the same as ESET's LiveGuard/EDTD. For most harmful samples, CyberCapture won't send them as a whole to its cloud for analysis; only a few suspicious ones will be sent. As a result, in my testing, many dangerous samples (for example, MBR killer) were rated as "safe to open" by local heuristics, without being sent to the cloud.

LiveGuard is different, everything unknown will be sent to the cloud sandbox for analysis.

In short, the threshold for sending files to the cloud differs, and CyberCapture relies more on its local heuristic engine and LiveGuard relies solely on its cloud sandbox.

I also want to mention another cloud-based detection technology from Avira - Avira Protection Cloud (APC). APC also uses local heuristics when deciding whether or not to send a file to the cloud. It primarily uses cloud-based heuristics to reach a decision in the cloud, allowing it to display results in seconds. It outperforms the above two in terms of detection rate in my testing.
 
Last edited:

Trooper

Level 17
Verified
Top Poster
Well-known
Aug 28, 2015
800
What I want to emphasize is that Avast's CyberCapture is not the same as ESET's LiveGuard/EDTD. For most harmful samples, CyberCapture won't send them as a whole to its cloud for analysis; only a few suspicious ones will be sent. As a result, in my testing, many dangerous samples (for example, MBR killer) were rated as "safe to open" by local heuristics, without being sent to the cloud.

LiveGuard is different, everything unknown will be sent to the cloud sandbox for analysis.

I also want to mention another cloud-based detection technology from Avira - Avira Protection Cloud (APC). APC also uses local heuristics when deciding whether or not to send a file to the cloud. It primarily uses cloud-based heuristics to reach a decision in the cloud, allowing it to display results in seconds. It outperforms the above two in terms of detection rate in my testing.

Good to know about Avira Protection Cloud. I am not really happy with ESET with their cloud analysis at my job. I may have to check their consumer suite out again at some point. It's been years since I have ran it.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,458
Why? What changed that you now doubt their validity?


I've never understood the logic in AV vendors coding their products to perform fewer checks on files not sporting MOTW. It just comes across as vendors deliberately hampering their ability to detect malware pre-execution.
The logic is simple. WIndows and AV vendors use MOTW to recognize if the file is downloaded from the Internet and use enhanced protection for such files (SmartScreen, CyberCapture, etc.). Other files are ignored, so the number of false positives is much lower.
If the application makes an update the updater is downloaded without MOTW. If the enhanced security was applied to all files, then many application auto updates could be blocked, especially when enhanced security is based on file reputation (like SmartScreen).
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,458
...
For most harmful samples, CyberCapture won't send them as a whole to its cloud for analysis; only a few suspicious ones will be sent. As a result, in my testing, many dangerous samples (for example, MBR killer) were rated as "safe to open" by local heuristics, without being sent to the cloud.
...
Interesting observation - worth to be tested.
What happened after running these samples recognized as "safe to open"? Did they manage to infect the system?
 

Anthony Qian

Level 10
Verified
Well-known
Apr 17, 2021
453
Interesting observation - worth to be tested.
What happened after running these samples recognized as "safe to open"? Did they manage to infect the system?
When CyberCapture has marked a file as "safe to open," the file will open automatically. Then, if IDP, Ransomware Protection, AMSI, exploit prevention and other relevant components are able to block this threat, they will. However, none of the above components can identify and block threats like MBR Killer. As a result, the computer was locked......
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,458
When CyberCapture has marked a file as "safe to open," the file will open automatically. Then, if IDP, Ransomware Protection, AMSI, exploit prevention and other relevant components are able to block this threat, they will. However, none of the above components can identify and block threats like MBR Killer. As a result, the computer was locked......
Are you sure that the sample has got MOTW? The malware samples are usually downloaded in Zip archives and after unpacking, the MOTW can be lost. No MOTW no CyberCapture. Furthermore, CyberCapture can work only for files with the .exe extension.

Edit.
By CyberCapture I mean CyberCapture with sandbox analysis.
 
Last edited:

Anthony Qian

Level 10
Verified
Well-known
Apr 17, 2021
453
Are you sure that the sample has got MOTW? The malware samples are usually downloaded in Zip archives and after unpacking, the MOTW can be lost. No MOTW no CyberCapture. Furthermore, CyberCapture can work only for files with the .exe extension.
Good to know. But in my testing, samples that have been upzipped can trigger CyberCapture. I can see the CyberCapture scanning interface after double-clicking.

Yes. MBR killer is a PE file and has .exe extension.
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,458
Good to know. But in my testing, samples that have been upzipped can trigger CyberCapture. I can see the CyberCapture scanning interface after double-clicking.

Yes. MBR killer is a PE file and has .exe extension.
What unpacker do you use?

In my tests with Avast, the sample without MOTW was recognized as suspicious and the alert about the scan was visible.

1650312032477.png


The same sample with MOTW triggered the above alert and additionally the below alert:

1650311882870.png
 
Last edited:

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top