AV-TEST Cybersecurity: Defense Against the Latest Attacking Techniques in the ATP Test (February 2024)

[Gandalf_The_Grey]

In an ongoing race against cybercriminals, security vendors need to constantly maintain the upper hand in order to sustainably guarantee the security of data for both consumer users and corporate users. The Advanced Threat Protection test from AV-TEST relies on detailed individual tests to examine whether the vendors are able to detect and defend against the latest, most sophisticated cyberattacks. Twenty-five products were evaluated on Windows systems in this test using ten scenarios to simulate ransomware and data stealer attacks on the systems. Special attacking techniques such as reflective code loading and fileless malware, which challenge modern security algorithms as they have to detect dangerous lines of code or scripts, were used. The outcome of the testing shows that overall the security products can defend their leading position; however, some products do not have all attack steps under control.

ATP test: results for consumer user products

The lab tested 12 end-user products in the extended ATP test to see how well they detect and defend against data stealers and ransomware using the latest cyberattack techniques. Products from the following vendors were put to the test: Avast, AVG, Avira, Bitdefender, ESET, F-Secure, G DATA, Kaspersky, Microsoft, Microworld, Norton, and PC Matic.

8 of the 12 protection packages for Windows examined had no problems at all during the entire test in detecting the attackers and immediately stopping and isolating them in one of the first two steps: Avira, Bitdefender, ESET, G DATA, Kaspersky, Microworld, Norton, and PC Matic.

Microsoft Defender detected the attackers in the ten scenarios, but in one case with ransomware it could not initially stop further execution. The startup file was generated, but it was then prevented from being executed, so in the end the system was not encrypted. In one case, the points scored were halved for this reason. In general, Microsoft scored 33.5 out of 35 points in this test.

The issues for the products from Avast, AVG, and F-Secure were almost identical in the test. The products detected the attackers in two cases with data stealers and two cases with ransomware; however, they were initially unable to prevent them from taking further action. The defense mechanism was only triggered when the data was about to be extracted or encrypted, which was when the destructive component was isolated and rendered harmless. It prevented data from being stolen and nothing could be encrypted.

Nevertheless, with the products from Avast, AVG, and F-Secure, the attackers managed to advance further than they should have been able to. For this reason, based on the four cases, there was a significant point deduction. At the end of the test, all of the three products mentioned received 29 out of 35 points for their protection score.

All protection packages earned the “Advanced Certified” certificate in the ATP test. The only exception here was G DATA: although the product performed well in testing, AV-TEST only certifies products that achieve certification in the regular monthly tests and fulfill all their criteria.

ATP test: results for endpoint products

The testing of corporate solutions examined endpoint products from the following vendors: Avast, Bitdefender (two versions), Check Point, ESET, HP Security, Kaspersky (two versions), Microsoft, Qualys, Seqrite, Symantec, and WithSecure.

The corporate product test went extremely well for nearly all vendors. 12 of the 13 endpoint products tested did not allow ransomware attackers or data stealers a chance in any of the ten scenarios, effectively stopping all attacks immediately. For this feat, all products received the full 35 points in terms of the protection score.

Seqrite was the only product that encountered a problem: it detected the attackers in two ransomware attacks and two data stealer attacks, yet it was unable to stop the initial actions. It was only possible in later steps for the product to isolate the malware and stop the attackers’ destructive efforts. In the end, no data was stolen or encrypted. Nevertheless, it hurt Seqrite in the scoring, leaving it with only 29 out of 35 possible points.

Cybersecurity: Defense Against the Latest Attacking Techniques in the ATP Test

In an ongoing race against cybercriminals, security vendors need to constantly maintain the upper hand in order to sustainably guarantee the security of data for both consumer users and corporate users. The Advanced Threat Protection test from AV-TEST relies on detailed individual tests to...

www.av-test.org

 

[Bot]

The ATP test from AV-TEST evaluated 25 security products on their ability to detect and defend against sophisticated cyberattacks. The test included scenarios simulating ransomware and data stealer attacks. For consumer products, 8 out of 12 had no issues detecting and isolating the attackers. Microsoft Defender, Avast, AVG, and F-Secure faced some challenges but managed to prevent attacks in the end. All products received an "Advanced Certified" certificate, except G DATA. For endpoint products, nearly all successfully stopped all attacks, except Seqrite, which had some initial difficulties but ultimately prevented data theft or encryption.

 

[Victor M]

All the techniques either use jscript or Powershell .( Read the info on each Scenario accompanying any vendor ) I never use Powershell nor jscript. So I block them. Your AV should be configurable to do the same. ( If not, then you should choose a product that has configurable rules ) If your AV has a Enable switch, then you can selectively disable the Block for instances when you do need them. There is Nothing in Windows that uses Powershell nor jscript, at least not in Settings and Control Panel, so you won't disrupt Windows in any way.

The only benefit when you employ those specific AV's mentioned in the test, is that they Might be able to remove the thing; depending on how they recognize it. Do your hardening and these attacks will be stopped. So what if an artifact remains, it is rendered harmless.

Most of the scenarios start off with a download from email attachment. So good secure habits plays a Very important part - you will be able to stop the attacks at stage 1. But, email attachments is only the Current favorite delivery method; and new methods will be discovered. So hardening is an important 2nd security layer that would stop the attacks, if stage 1 succeeds.

 

[Practical Response]

Most of the scenarios start off with a download from email attachment. So good secure habits plays a Very important part - you will be able to stop the attacks at stage 1. But, email attachments is only the Current favorite delivery method; and new methods will be discovered. So hardening is an important 2nd security layer that would stop the attacks, if stage 1 succeeds.

Watch those responses because I have caught much flak for stating the very same thing although it's absolutely real world.

 

[Andy Ful]

There is a problem with such tests. They simulate the most popular techniques used by attackers in the past. But in the times of Malware-as-a-service, those techniques are replaced by others poorly detected by AVs. So in the tests, the AVs look good. But the reality is slightly different.
In all tested scenarios, the encoded PowerShell script was written to the registry and was used to communicate with the C2 server. It is a well-known method used in the wild for several years. Most AVs can effectively use AMSI-based detections to stop such attacks at the Execution stage (if the malware is undetected at the Initial Access stage ).
The widespread attacks may look possibly like that (I am not sure), but I think that the scenarios used for the Enterprise AVs could be more comprehensive by adding some other popular methods with batch scripts, HTA files, DLL hijacking, etc.

In the home environment, all attacks could be simply mitigated by blocking outbound connections of PowerShell (Command and Control stage).
Of course, such a simple mitigation is not sufficient for many other fileless techniques.

 

[Practical Response]

There is a problem with such tests. They simulate the most popular techniques used by attackers in the past. But in the times of Malware-as-a-service, those techniques are replaced by others poorly detected by AVs. So in the tests, the AVs look good. But the reality is slightly different.
In all tested scenarios, the encoded PowerShell script was written to the registry and was used to communicate with the C2 server. It is a well-known method used in the wild for several years. Most AVs can effectively use AMSI-based detections to stop such attacks at the Execution stage (if the malware is undetected at the Initial Access stage ).
The widespread attacks may look possibly like that (I am not sure), but I think that the scenarios used for the Enterprise AVs could be more comprehensive by adding some other popular methods with batch scripts, HTA files, DLL hijacking, etc.

View attachment 283003

In the home environment, all attacks could be simply mitigated by blocking outbound connections of PowerShell (Command and Control stage).
Of course, such a simple mitigation is not sufficient for many other fileless techniques.

This testing is accurate in that its in the wild malicious items and testing with realworld routes of infection, not POCs with half baked tests. Fileless malware are not new and have been around prevalent since around 2017. IOA "indicators of attack" being necessary over IOC "indicators of compromise" in these suites is something I think they are understanding by now. Of course delivery has everything to do with how the malicious item affects a system regardless of what type it is, and as shown, most of these are introduced via Phishing campaigns.

 

[Andy Ful]

This testing is accurate in that its in the wild malicious items and testing with realworld routes of infection, not POCs with half baked tests.

So it could be as correct as any malware test with 10 in the wild samples.
By the way, the samples were custom-made by AV-Test and only simulated real-world scenarios in a limited way.

 

[Practical Response]

So it could be as correct as any malware test with 10 in the wild samples.
By the way, the samples were custom-made by AV-Test and only simulated real-world scenarios in a limited way.

It's 9 more samples then another test you supported here recently. 5&5 of ransomware & infostealers of which reflective code injection and fileless methods were incorporated.

All attack scenarios are documented according to the standard of the MITRE ATT&CK database.