AV-TEST ATP Test: How easily Windows can be tricked by malware

[SeriousHoax]

Many users have known for a long time that they always need to keep their Windows up to date so that cyberattackers will have as few opportunities as possible to launch an assault. But Windows itself has a number of vulnerabilities that only security software can mitigate. Interesting fact: Windows attempts to load DLL files even if they are only referenced in code and the actual file does not even exist. Attackers create these phantom files, including malware code, and make them available to a Windows process. This process becomes an unintentional facilitator of the ransomware or infostealer attack. In the latest Advanced Threat Protection test – ATP test for short – 19 protection products for consumer users and corporate users demonstrate whether they see through all these DLL guises, including malware, or whether they fall for them..........................................
....................................................................................................................................................................

19 security products in the ATP test under Windows 11​

All products were required to demonstrate in 10 real attack scenarios that they identified the malware and were capable of fending it off in further steps. Each additional action is documented in the ATP test and illustrated in the results graphs.

In test lineup were 10 consumer user products from Avast, AVG, Avira, ESET, F-Secure, G DATA, Kaspersky, McAfee, Microsoft and Norton. The solutions for corporate users came from the vendors Acronis, Avast, Bitdefender, ESET, Kaspersky (with two versions), Microworld, Qualys and Trellix.

In each ATP test, the experts in the lab deploy alternating attack techniques, just as cybergangsters do in real life. In the 10 scenarios involving 5 samples of ransomware and infostealers each, the testers award a specified number of points for the performance of each product. This means up to 3 points for ransomware, and up to 4 points for infostealers. Half points are also awarded if a product is capable of mounting a partial defense against an attack. At the end of the test, each product is ultimately able to earn up to 35 points for its protection score.

In this test, two special attack techniques were used:.................................................................

ATP Test: How easily Windows can be tricked by malware

Many users have known for a long time that they always need to keep their Windows up to date so that cyberattackers will have as few opportunities as possible to launch an assault. But Windows itself has a number of vulnerabilities that only security software can mitigate. Interesting fact...

www.av-test.org

 

[SeriousHoax]

From the article:

The products examined for consumer users came from the following vendors: Avast, AVG, Avira, ESET, F-Secure, G DATA, Kaspersky, McAfee, Microsoft and Norton. The test proceeded perfectly for 7 of the 10 products and they were not to be fooled by any attack technique. For this they all received the maximum 35 points for their protection score.

The ESET package identified 10 out of 10 attackers. However, encountering a ransomware sample, ESET identified, but was not able to completely stop the attacker. The insertion of an infected DLL file was also not prevented. In the end, data was encrypted and 2 out of 3 possible points were lost. ESET finished the test with 33 out of 35 points.

The packages from Microsoft and G DATA identified and fended off 9 attackers without errors, but both fell short when it came to an infostealer. No detection, no blocking and no further defense mounted – all the data was stolen. Both products thus lost 4 points each, leaving them with 31 out of 35 points for the protection score.

 

[Parkinsond]

Avast free is as good as K premium, ESET ultimate, and Norton 360, followed by MD.

 

[Khushal]

ATP Test: How easily Windows can be tricked by malware

Many users have known for a long time that they always need to keep their Windows up to date so that cyberattackers will have as few opportunities as possible to launch an assault. But Windows itself has a number of vulnerabilities that only security software can mitigate. Interesting fact...

www.av-test.org

The ESET package identified 10 out of 10 attackers. However, encountering a ransomware sample, ESET identified, but was not able to completely stop the attacker. The insertion of an infected DLL file was also not prevented. In the end, data was encrypted and 2 out of 3 possible points were lost. ESET finished the test with 33 out of 35 points.

 

[Khushal]

Avast free is as good as K premium, ESET ultimate, and Norton 360, followed by MD.

maybe even better than eset according to this test

 

[Parkinsond]

maybe even better than eset according to this test

True

The ESET package identified 10 out of 10 attackers. However, encountering a ransomware sample, ESET identified, but was not able to completely stop the attacker.

 

[Divergent]

Technical Analysis​

The test utilized two primary MITRE ATT&CK-aligned techniques.

Phantom DLL Hijacking (T1574.001)
Exploiting Windows services (e.g., SessionEnv) that search for non-existent DLLs (e.g., TSVIPSrv.dll).

DLL Sideloading (T1574.002)
Using legitimate binaries (e.g., Microsoft Edge's identity_helper.exe) to load a malicious version of a required library.

​

Top Performing Products​

Consumer Sector (35/35 Points)
Avast, AVG, Avira, F-Secure, Kaspersky, McAfee, and Norton.

Enterprise Sector (35/35 Points)
Acronis, Avast, Bitdefender, ESET, Kaspersky (two versions), Microworld, and Trellix


Why they succeeded
These products utilized Behavioral Monitoring and EDR/HIPS logic to recognize the execution of ransomware or infostealer code even when it was initiated by a trusted system process. They did not rely solely on the "reputation" of the parent process, but rather inspected the actual API calls and system changes (e.g., encryption attempts) made by the hijacked service.

Standard industry practice for AV-TEST comparative evaluations (which this report belongs to) typically involves testing products with their "out-of-the-box" configurations to reflect the experience of a standard user. The report emphasizes the role of additional, built-in protection modules that fended off attackers even when initial detection failed, suggesting these were standard features of the evaluated software.


While the AV-TEST report uses simulated scenarios, the underlying techniques, DLL Sideloading and Phantom DLL Hijacking, are among the most prevalent execution vectors used by modern malware families like LummaStealer, Vidar, and Black Cat.

Threat Profile​

Home Users
Most commonly encounter these via malicious downloads (pirated software, "cracks," or fake updates) and phishing emails with attachments like ISO files or ZIP archives.

Enterprise/POCs
While frequently used in "Red Team" penetration tests to demonstrate how to bypass EDR, these are daily occurrences in incident response cases. Organizations are particularly vulnerable through third-party software (supply chain risk) that has insecure DLL loading paths.

 

[SeriousHoax]

True

ESET missed a ransomware, MD missed an infostealer.
Pick your poison.
ESET MD

 

[Parkinsond]

ESET missed a ransomware, MD missed an infostealer.
Pick your poison.
ESET MD

So the winners for home use are K and Avast/Norton.

 

[Khushal]

True

In the end, data was encrypted and 2 out of 3 possible points were lost

Generous of them to give them even one point. File encryption should get negative points in my opinion.

 

[Jonny Quest]

So the winners for home use are K and Avast/Norton.

And F-Secure, regarding this test.

 

[SeriousHoax]

And F-Secure, regarding this test.

Also Avira and McAfee.

 

[Khushal]

Avast free is as good as K premium, ESET ultimate, and Norton 360, followed by MD.
interesting behavior by avast free when compared to paid norton and avg on one file.

 

[Trident]

McAfee is the perfect all-rounder that consumers will install rather quickly (30s-1min) and easily, and will forget. McAfee on this test has blocked every attack very early, nothing progressed even to the second stage.

Eset and Microsoft Defender are all about the configuration, however Eset configurations are much more convoluted.

All 3 rely primarily on user-mode monitoring, showing that modern and stable architecture can achieve the same efficiency (if not higher) as all other architectures nested deeply in the Windows kernel.
This design will slowly become the default one for many solutions.

Solutions like Kaspersky and F-Secure also fit within the all-rounder group. With Kaspersky you have the granular configuration when you want it, but when you don’t want it (or don’t understand it), out of the box the solution is more than efficient.

 

[Jonny Quest]

McAfee is the perfect all-rounder that consumers will install rather quickly (30s-1min) and easily, and will forget. McAfee on this test has blocked every attack very early, nothing progressed even to the second stage.

Eset and Microsoft Defender are all about the configuration, however Eset configurations are much more convoluted.

All 3 rely primarily on user-mode monitoring, showing that modern and stable architecture can achieve the same efficiency (if not higher) as all other architectures nested deeply in the Windows kernel.
This design will slowly become the default one for many solutions.

Solutions like Kaspersky and F-Secure also fit within the all-rounder group. With Kaspersky you have the granular configuration when you want it, but when you don’t want it (or don’t understand it), out of the box the solution is more than efficient.

I've seen this in other tests, where F-Secure (maybe Avira, too?) will block it on execution. What is the difference between why some are better at initial access and others at execution? What is the Avira SDK "missing" to be able to catch things earlier?

 

[anirbandutta01]

I'm disappointed to see ESET's result. It's time to try McAfee or use my Avast Antivirus Pro.

 

[Khushal]

I'm disappointed to see ESET's result. It's time to try McAfee or use my Avast Antivirus Pro.

ESET's performance against ransomware has always been below industry standard for many years. This was just another poor showing.

 

[Antimalware18]

Nice showing, it just doubles down in my mind that with MD configuration is key, as well as having a Firewall that notifies or is sufficiently hardened.
So while the infostealer might be present, it cant relay any data to C&C servers.

 

[Parkinsond]

I like pre-execution block.

 

[Jonny Quest]

Nice showing, it just doubles down in my mind that with MD configuration is key, as well as having a Firewall that notifies or is sufficiently hardened.
So while the infostealer might be present, it cant relay any data to C&C servers.

I agree, as far as a firewall notification, "pre-execution"