AV-TEST Advanced Endpoint Protection: Ransomware Protection test (commissioned by Kaspersky)

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Gandalf_The_Grey

Level 53
Verified
Trusted
Content Creator
Apr 24, 2016
4,242
41,364
In June-August 2021, AV-TEST carried out a test of ransomware protection offered by 11 different Endpoint Protection Platforms (EPP). In total, 113 different attacks were executed.

The three assessment scenarios were independently developed and executed by the test lab:
• Real-World ransomware attacks user files on local system
• Real-World ransomware attacks user files on remote shared folder
• Proof of Concept ransomware attacks user files on local system

During the test, the products were expected to detect ransomware activity and its files, block it, roll-back any changes to user files (the other words, to protect all user files) and eliminate the threat from the targeted system. Only these results were considered a true success and the relevant solution was given a credit in each test case.

Schermafbeelding 2021-10-05 155812.png

Kaspersky Endpoint Security Cloud achieved the best results, protecting against 100% of all the ransomware attacks in the test (113 in total), without loss of a single user file.

The individual results of the three scenarios revealed a difference in the detection/protection capabilities of the products being tested

Kaspersky blog post:
AV-Test results in pdf:
 

tipo

Level 7
Jul 26, 2012
306
723
Kaspersky is definitely good at blocking ransomware. But I take this test result with a grain of salt as this test is commissioned by Kaspersky.
I don't think you should. If you watch even the test against ransomware of that tpcsc guy, disabling almost all the shields of kis, you would be impressed of the results kis had. Comissioned or not by kaspersky, it really has (probably) the best ransomware protection out there. My 2 cents.
 

carl fish

Level 6
Verified
Mar 6, 2012
298
800
I don't think you should. If you watch even the test against ransomware of that tpcsc guy, disabling almost all the shields of kis, you would be impressed of the results kis had. Comissioned or not by kaspersky, it really has (probably) the best ransomware protection out there. My 2 cents.
and their interface and functionality for their small office security is almost the same as their consumer products
 
Last edited:

Anthony Qian

Level 3
Apr 17, 2021
149
757
I don't think you should. If you watch even the test against ransomware of that tpcsc guy, disabling almost all the shields of kis, you would be impressed of the results kis had. Comissioned or not by kaspersky, it really has (probably) the best ransomware protection out there. My 2 cents.
I've already said:
Kaspersky is definitely good at blocking ransomware.
The question is that whether the other products tested are truly that bad at dealing with ransomware. To be honest, I am unpleasantly surprised at Bitdefender's performance.

Because this test was commissioned by Kaspersky, I believe it can fully or partially determine the test samples and methods, either directly or indirectly. So it can select its own favorable test samples and testing methods.
 
Last edited:

MacDefender

Level 14
Verified
Oct 13, 2019
699
6,589
Two things strike me about this test:

• Proof of Concept ransomware attacks user files on local system

POC ransomware would be mostly a behavior blocker test and it’s not surprising that KSW is one of the best in the industry. Some of the poor performers like ESET, Microsoft Defender, and Symantec are not super surprising as they’ve not done well against most other proof of concept samples either.

During the test, the products were expected to detect ransomware activity and its files, block it, roll-back any changes to user files

This requirement around rolling back user files is a key feature of KSW and I don’t believe many of the other products support a rollback mechanism. I suspect this is where a lot of other behavior blockers lost points.

Enterprise ransomware attacks tend to be challenging because the point of entry and lateral traversal tends to be harder to identify. On your home computer, pretty much everything untrusted will arrive through your web browser or a removable drive and security products are tuned to be extra paranoid about that. In an enterprise setup there’s simply too many ways to inject new files into endpoints and most tend to be legitimate. Symantec’s forums show more complaints about legit login PowerShell scripts getting blocked than anything else.
 

ExecutiveOrder

Level 1
Verified
Sep 21, 2021
27
140
If this test is also quite specific to APT attacks like AV-Comparatives "Enhanced Real World" test, Kaspersky is also ahead compared to others, while something like Microsoft chose to opt-out, but the difference isn't like this 'weird' (0%, 21%, 36%, 50%, 86%, 100%).
and their interface and functionality for their small office security is almost the same as their consumer products
What about Kaspersky Endpoint Security Cloud which was used in the test? It's intended for IT admin to manage organizational protection unlike KSOS non-IT admin and protection monitoring (install and forget). I think it's still quite similar but not as similar as KSOS to consumer products.

Because this test was commissioned by Kaspersky, I believe it can fully or partially determine the test samples and methods, either directly or indirectly. So it can select its own favorable test samples and testing methods.
AV-Test claims that "three assessment scenarios were independently developed and executed by the test lab", "results were independently verified", and "samples were selected independently, right before the test execution from real-time sources". But for me, it still smells like ocean saltwater.

This requirement around rolling back user files is a key feature of KSW and I don’t believe many of the other products support a rollback mechanism. I suspect this is where a lot of other behavior blockers lost points.
AV-Test regarding "expected requirements" including roll-back specifically for encrypted user files:
Only test cases where 100% of user files were protected by the security solution were considered a success, and represented as “Completely blocked malware attacks”. This means that in the rest of the test cases, security solutions turned out to be tolerant to sacrificing user files, up to 100%. Considering that any single encrypted file could be critically important for a user and the business, anything less than 100% protection against ransomware is unacceptable.
I also believe this causes a wide gap in protection rate because this is about ransomware and its impact on user files, it is just like testing malware protection with only "protection clusters/groups" without an overall "protection rate" (AV-Comparatives did use a table of comprehensive protection rate across different scenarios in general "Malware Protection Test" though is not ransomware-specific test.

Not sure if other vendors were notified about this prior to the test (or even prior preparation of the test from December 2020) and have a chance to opt-out like AV-Comparatives brand new (performed twice since 2019) APT test, not all "Business Windows Client" vendor were here (11 vendors of 18 total in latest AV-Test certification report).
If not, I really want to hear what they think and really wish this kind of special test will be performed at least annually by AV-Test without the need for a commission or request from a certain vendor in the future.
 

MacDefender

Level 14
Verified
Oct 13, 2019
699
6,589
AV-Test regarding "expected requirements" including roll-back specifically for encrypted user files:
I also believe this causes a wide gap in protection rate because this is about ransomware and its impact on user files, it is just like testing malware protection with only "protection clusters/groups" without an overall "protection rate" (AV-Comparatives did use a table of comprehensive protection rate across different scenarios in general "Malware Protection Test" though is not ransomware-specific test.
Ah I missed reading this statement about 100% rollback being required. I don't disagree with their reasoning, but I will say that when I've tested proof-of-concept homebrew ransomware, it was pretty common for their behavior blocker to trigger after 1 or 2 documents got encrypted. Definitely kudos to KSW for having a rollback architecture, very few other products bother to implement the complexity of backing up files.
 

Andy Ful

Level 73
Verified
Trusted
Developer
Dec 23, 2014
6,284
42,889
For non-enterprise users, the most important thing is:
1633703598477.png

The samples were selected independently, right before the test execution from real-time sources.

Ransomware samples from the following 20 real-world ransomware families were selected for this scenario: conti, darkside, fonix, limbozar, lockbit, makop, maze, medusa (ako), mountlocker, nefilim, netwalker (aka mailto), phobos, PYSA (aka mespinoza), Ragnar Locker, ransomexx (aka defray777), revil (aka Sodinokibi or Sodin), ryuk, snatch, stop, wastedlocker


The interpretation of protection results in the enterprise environment would be more complicated.
It is true that Kaspersky has got one of the best anti-ransomware protection when the ransomware is executed on the already compromised system. But this does not mean that other products are far behind, because many of them can prevent the attacks before ransomware could be delivered/executed.
 

Shadowra

Level 5
Sep 2, 2021
249
2,781
Kaspersky <3

I really love this Russian AV payer that protects my whole family.

Its result does not surprise me very much, Kaspersky has an excellent Cloud + BB to block Ransomware. One of the few to detect mine in the middle of coding :D
On the other hand, I'm very surprised to see that the others are around 60% when they are all equipped with anti-ransomware and other modules....
 

Plag

Level 2
Apr 15, 2020
87
275
Kaspersky <3

I really love this Russian AV payer that protects my whole family.

Its result does not surprise me very much, Kaspersky has an excellent Cloud + BB to block Ransomware. One of the few to detect mine in the middle of coding :D
On the other hand, I'm very surprised to see that the others are around 60% when they are all equipped with anti-ransomware and other modules....
I see your security config, but u dont use the Kaspersky. Why?
 

Plag

Level 2
Apr 15, 2020
87
275
Having tested Overwatch and Need For Speed, I did not notice any slowdowns (even though I am on Windows 11).

On Call of Duty, I had to slow down a bit, but I suspected it ;)
Kaspersky slow down the performance when u play with Call of Duty?
 

ExecutiveOrder

Level 1
Verified
Sep 21, 2021
27
140
Has anyone here ever used or evaluate any of the enterprise products that scored 0% in Test scenario #2 there?
Just curious, do they (with default settings) are 'extensively' monitors the remote shared folders during a cyber attack especially ransomware?
I wonder if there's a simple switch that could significantly improve the protection against ransomware targeting shared folders because they probably are not configured to do so by default.
All the products were tested with their default configuration.
 
Last edited:
Top