AV-Comparatives Advanced Threat Protection Test 2021 – Consumer

[Andy Ful]

If you look at the Detection/Blocking stages table, you will find Avast (AVG) does not simply block all obfuscated scripts. In case 7 and 10, Avast blocked immediately after the threat has been run. In case 9, Avast blocked after the threat has been run, and its actions have been recognised.

The case 7 behaves just like Avast blocked obfuscated script run via Office macro.
The case 10 has nothing to do with scripts.
In the case 9 the CmdLine in .lnk file managed to download the PowerShell payload (obfuscated script) and the payload was blocked by Avast.

Edit.
There is an error in the online report (wrong numeration of scenarios). So my answer about scenarios 9 and 10 is in fact about scenarios 10 and 11. The corrected answer is here:
https://malwaretips.com/threads/advanced-threat-protection-test-2021-–-consumer.110873/post-964448

 

[Andy Ful]

AVG (and probably Avast) simply blocks all obfuscated scripts and this helped to get the best result.

Among 15 testing scenarios, there is an interesting sample nr 14 (corrected numbering) that managed to bypass the protection of 4 AVs. In the case of Avast, it was most probably blocked by Cyber Capture.
The attack was performed via spearphishing link, so the EXE file (masquerading werfault.exe) was downloaded/executed (MOTW added by the web browser). The MOTW is required for an EXE file to trigger Cyber Capture, so the file was uploaded and detonated in the cloud sandbox.

Another interesting example is sample nr 12 (corrected numbering). This sample could bypass 5 AVs (including Avast). The Avast CyberCapture could not block it. This attack is similar to scenario 14 (which was blocked by Avast). There is a note about this in the report:

Avast, AVG: In one case, the Sandbox came back with the verdict that the file was safe. Then, while the threat was already running in memory, it was detected, but a stable C2 connection remained open anyway, and the attack continued without restrictions.

Edit.
Post edited to include the corrections due to scenario numbering error in the online report.
I removed the note about scenario 12 (incorrect numbering) because, in fact, it was about scenario 13 (correct numbering). This scenario was blocked by Avast (script was blocked). If this script was not blocked then CyberCapture could not detect the DLL sideloading technique.

 

[The_King]

Is it possible for someone from MT (Malware Hunters) to contact AV-comparatives and ask for these samples to test in the Malware hub?

 

[Andy Ful]

Is it possible for someone from MT (Malware Hunters) to contact AV-comparatives and ask for these samples to test in the Malware hub?

All will be probably detected by all vendors. But, one could probably modify them for further testing.

 

[Gandalf_The_Grey]

According to IBK from AV-Comparatives Bitdefender does scan scripts by default:

they scan also scripts by default.

AV-Comparatives: Advanced Threat Protection (ATP) Test 2021

AV-Comparatives has released the results of its 2021 Advanced Threat Protection Tests. Eight consumer-antivirus products and eight enterprise...

www.wilderssecurity.com

 

[The_King]

According to IBK from AV-Comparatives Bitdefender does scan scripts by default:

AV-Comparatives: Advanced Threat Protection (ATP) Test 2021

AV-Comparatives has released the results of its 2021 Advanced Threat Protection Tests. Eight consumer-antivirus products and eight enterprise...

www.wilderssecurity.com

I have both BIS and BTS subs.
I can confirm its off by default. I clicked reset settings and the option is disabled

 

[Andy Ful]

Probably they're not confident about their product doing well in this test.

I found this in the report:

However, precisely the same product and configuration is used for all the tests in the series.

When we look into the Business Security Test March-June 2021 (full report) we can see for Defender:

Microsoft: Google Chrome extension “Windows Defender Browser Protection” installed and enabled; “CloudBlockLevel” set to “High”.

It means that Microsoft is tested without ASR rules and scored very well. This can be the reason why Microsoft does not want to participate in Advanced Threat Protection tests that would require enabling ASR rules.

 

[SumTingWong]

ESET

 

[The_King]

I was really disappointed with Bitdefender's result. It is clear that it does not scan scripts by default, but would it really improve the result? And why did they disable script protection in the default settings at all, if this is one of the most popular attack vectors?

There are a two things I have done to improve BD protection on my system.

1. Turn scan scripts on. (this may or may not have helped in the tests carried out here)
2. Under Firewall Turn on Alert mode. This will be a slight annoyance at first however after a day or two
it will only alert you if a new program or a changed/updated program is trying to connect to the internet.

I only get alerts now after Edge and Firefox update or after doing a windows security update.
This should improve protection from default.

Also keep in mind this is one set of tests and does not mean the product is completely usless.
Some products that scored well here do not score has well in other malware related tests carried out by the same site.

Edit.
I also blocked a few LOLbins in BD firewall from accessing the internet.

 

[Andy Ful]

If you look at the Detection/Blocking stages table, you will find Avast (AVG) does not simply block all obfuscated scripts. In case 7 and 10, Avast blocked immediately after the threat has been run. In case 9, Avast blocked after the threat has been run, and its actions have been recognised.

There is an error in the online report. Scenarios 9 and 10 are glued together and scenarios 10-14 are in fact 11-15. The PDF report is OK.
It seems that in scenario 10 (after correction) Avast blocked the non-obfuscated VBS script.
In scenario 9 the CmdLine in .LNK shortcut was used to download an obfuscated PowerShell payload which was blocked by Avast.

 

[Andy Ful]

There are a two things I have done to improve BD protection on my system....

There is a simpler and safer solution. All scenarios could be blocked if you would use Bitdefender + block scripts + respect SmartScreen. The same is true for most AVs included in the test.

 

[Gandalf_The_Grey]

I have both BIS and BTS subs.
I can confirm its off by default. I clicked reset settings and the option is disabled
View attachment 261855

From IBK:

Instead of looking at that setting (which also here is disabled, as it is all default with consumer products), please try malicious scripts and look whether they are scanned (they are, otherwise it would have missed several more).

AV-Comparatives: Advanced Threat Protection (ATP) Test 2021

AV-Comparatives has released the results of its 2021 Advanced Threat Protection Tests. Eight consumer-antivirus products and eight enterprise...

www.wilderssecurity.com

 

[The_King]

From IBK:

AV-Comparatives: Advanced Threat Protection (ATP) Test 2021

AV-Comparatives has released the results of its 2021 Advanced Threat Protection Tests. Eight consumer-antivirus products and eight enterprise...

www.wilderssecurity.com

I am on Wilders seen his reply.
I have the same handle there.
I am not a malware tester so not going to run malcious scripts on my PC to test.

 

[SeriousHoax]

I am on Wilders seen his reply.
I have the same handle there.
I am not a malware tester so not going to run malcious scripts on my PC to test.

BD scans and block scripts just fine in the default settings after execution by signatures as well as by the behaviour blocker. But I don't know what extra thing it does when that option is enabled.

 

[ExecutiveOrder]

There are some interesting points to note in their testing.
By default BD does not scan scripts.
Would BD have blocked these scripts if scan scripts option was turned on?

Not sure if Bitdefender changed its product behavior regarding script detection in the past 2 years.

It's a bit late to reply, but...
In ATP 2020 test, with the same "default" settings, it successfully blocked a similar "script" threat to number (3):

2) This threat is introduced via Trusted Relationship. A PowerShell script containing an AMSI bypass and a PowerShell Empire stager was executed.

In ATP 2019 test, it successfully blocked a "script" threat scenario:

9) This threat is introduced via Removable Media (USB). A PowerShell script executes a PowerShell payload into memory. This test case was created with Unicorn.

But also failed to another "script" threat scenario:

15) This threat is introduced via Spearphishing Link. A PowerShell script injects an obfuscated PowerShell payload into memory. This test case was created with Metasploit Meterpreter.

I have both BIS and BTS subs.
I can confirm its off by default. I clicked reset settings and the option is disabled

Based on this user guide:

The Scan scripts feature allows Bitdefender to scan powershell scripts and office documents that could contain script-based malware.

Not sure if it means only applicable to scripts (including PowerShell) inside macro files (like in office documents), it didn't mention if this setting is turned on by default. None of the "script" test scenarios were using macros as an attack vector. Also, this user guide is similar to the website support information, published in late 2019, before ATP 2020 and 2021 tests were started.
In this Bitdefender community post, look like it is just an UI issue, the original post said that after installation it's OFF but turned ON after reset (contrary to your experience, 2 years apart though, different versions), another user replied that the feature didn't appear in UI and have to contact support based on moderator's suggestion.

Do it only in guest VM if you want to test it against any malicious script.
Anyway, at the end of the day, we can conclude it's turned ON based on a statement by AV-Comparatives themselves:

According to IBK from AV-Comparatives Bitdefender does scan scripts by default:

AV-Comparatives: Advanced Threat Protection (ATP) Test 2021

AV-Comparatives has released the results of its 2021 Advanced Threat Protection Tests. Eight consumer-antivirus products and eight enterprise...

www.wilderssecurity.com

Honestly, I also expect better from Bitdefender and am rather disappointed by these results, but anything can be turned out unexpected.

 

[SeriousHoax]

I will never use bitdefender again

I was trying out BD Free on Windows and was disappointed with it because of very high RAM usage on my low-end laptop and lack of AMSI support for scanning scripts. My experience with their Android product has been much better than their Windows product.

malwaretips.com

Thanks to user @South Park's comment on the above thread, I got interested and checked whether he's right or not. Turns out the "Scan Script" feature in Bitdefender which is off by default (for most if not all) is AMSI. When it's enabled, AMSI is on, otherwise it's off. I checked this using three methods in the latest Bitdefender Total Security 26.0.3.29.
So this could be the reason why Bitdefender performed worse than expected in this test. Bitdefender free have almost no configuration options, so AMSI is not present in that product. Looks like the feature being off in the paid versions is not a bug, it's by choice. Though I don't know why.
All paid Bitdefender user should manually enable the feature to increase protection against scripts.