Battle Ransomware simulator vs 10 AVs

Compare list
BitDefender Total Security 2022
WiseVector StopX v306
ESET Internet Security 15.1.2.0
Kaspersky Free (UK version) 21.5.11.384 (Patch B)
MS Defender
MS Defender Hardened
F-Secure Safe 18.5
Avira Prime
G-Data Total Security 2022
Avast Free

miguelang611

Level 2
Thread author
Apr 13, 2020
99
Environment:
VMWare
Windows 10 LTSC 21H2
Windows Defender fully disabled (except for Windows Defender tests ofc)
4 GB RAM
No Internet

Ransomware simulator KnowBe4


1. BitDefender Total Security 2022
Just Ransomware shield. ATP identifies the simulator, not the ransomwares itself, so we will just base the test on real behavioural shield, the ransomware remediation:
1649971890294.png

1649971941096.png


Test is pretty fast. About 1:30 min, and result is pretty good: 17/23 non vulnerable.
Resource usage however can be tremendous, having few moments when BD takes up over the ransomwares themselves.

1649972110943.png


------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


2. Eset Internet Security
1649972358906.png


1649972414437.png

8 and 9 keep stuck. If we have a look at their folders, we didn't get ransomed. However, I have made this test a couple of times and results differ... Usually they do get ransomed, but hey, we got lucky this time.
1649972916689.png

We got popups, but we didn't interact with them.
Note: with custom settings from RoboMan (Q&A - Configure ESET Antivirus for Maximum Security (by RoboMan)), you would be overwhelmed by pop-ups. Will depend if you allow or block... But nothing automatic. And yeah, I forgot to disable Internet, but results are same for Eset.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


3. WiseVector StopX v306
1649973164685.png

1649973186367.png


1649973251824.png

Depending on the ransom, it will autoremediate (16 as shown up here) or ask for recover (20 as shown down here):
1649973415383.png

Test ends up soon, also about 1:30. Real-time protection and advanced malware protection are disabled. We just leave ransomware shields enabled and with default settings.
Result: 11 non vulnerable.
Note: very light on resources, similar to ESET.
Note 2: great improvement from V305 (WiseVector StopX vs 0-day ransomware (KnowBe4)). On equivalent conditions we were just protected of 4 ransom and just few days after, we got 11!!


------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


4. Kaspersky Free 21.5
1649974126494.png

1649973778206.png

1649973837658.png

Let's disable AMSI and rely on system watcher:
1649974215285.png

1649974376277.png

1649974422287.png

Surpringsily I can't find which component remediated the behaviour:
1649974494524.png

Threats aren't loaded in memory, system is clean, but is it really remediated or is it preblocked? On the past I remember I got system watcher popups and I could see cxp loading, but not anymore...
Result: FULLY CLEAN


------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


5. F-Secure Safe 18.5
1649973943321.png

1649973962184.png

1649974307175.png

Results speak for themselves... 0 non vulnerable

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


6. Avira Prime
1649974975481.png

1649975030794.png

1649975074990.png

Note: Fastest test, although very bad results as well...


------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


7. Microsfot Defender Default:
1649975260000.png

1649975305885.png

1649975344500.png

Well done Defender!! 1 popup 1 protected!! No extra hassle!!

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


8. Hardened Microsoft Defender
Hardened Defender (security level: high/interactive/max - no difference)
1649976942087.png

1649976992977.png

With Interactive/Max, the script is blocked (similar behaviour as if we keep enabled signatures on BD/ESET or the 2 other engines in WVX). This is not ransomware remediation but raw blocking --> Not valid for the test:
1649977113541.png

1649977197919.png

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


9. G-Data Internet Security 2022
1649975623322.png

1649975651537.png

Very heavy on resources... Killed our test:
1649975697983.png

But only 1 ransom catched:
1649975727367.png

Let's reboot... Should we hope?
1649975896955.png

Nah, we shouldn't!!



------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


10. Avast Free
1649976107072.png

1649976160194.png

1649976188089.png

1649976492126.png
 

Attachments

  • 1649975990058.png
    1649975990058.png
    246.7 KB · Views: 189
  • 1649976440841.png
    1649976440841.png
    495.9 KB · Views: 192

miguelang611

Level 2
Thread author
Apr 13, 2020
99
Kaspersky proves once again its performance against Ransomware!

Too bad, no Norton in your test, I would have liked to see the SONAR in activity

With Norton

OsOk4MGP.png
When I first did these tests in the past Kaspersky was already the most advantaged pupil, both Kaspersky and BD improved however! And as far as I see as you tested, Norton does a pretty fair job as well!!
AVAST/AVG protect from ransomware only folders that have been specified by the user (excluding folders: documents, images, videos etc that are included by default)
Correct! I tried KnowBe4 on Documents folder and avast would prevent anything to run not trusted, so would have worked!
I have repeated the tests (Windows 10 Home 21H2), but now I started from Default settings and removed dynamic signatures after each test:

View attachment 265938
  1. Defender Default ---> 12 vulnerable, 2 false positives.
  2. Defender Default + Cloud Block Level = Highest ----> 5 vulnerable, 2 false positives.
  3. Defender Default + Cloud Block Level = Block ----> 0 vulnerable, 2 false positives.
Microsoft recommends Cloud Block Level = Highest, as a strong anti-ransomware protection.
As I wrote in my previous test, it is very probable that Defender learned much from post-execution telemetry compared to the test done by @miguelang611. This confirms my opinion that the results do not show the real capabilities of tested AVs, but rather show if someone else used KnowBe4 in the past to test the particular AV.
Well wow, that is a huge increase in protection from what I got. However I gotta say that I repeated these tests with 12 days between my first tests and when I took these with screenshots. They were pretty much equal between both timestamps... However I didn't try Defender (neither stock or hardened) on my first test (just the other 8 options) so that might be the reason!

Very useful comment!
 

Andy Ful

From Hard_Configurator Tools
Verified
Honorary Member
Top Poster
Developer
Well-known
Dec 23, 2014
8,040
I have repeated the tests (Windows 10 Home 21H2), but now I started from Default settings and removed dynamic signatures after each test:

View attachment 265938
  1. Defender Default ---> 12 vulnerable, 2 false positives.
  2. Defender Default + Cloud Block Level = Highest ----> 5 vulnerable, 2 false positives.
  3. Defender Default + Cloud Block Level = Block ----> 0 vulnerable, 2 false positives.
...

I am not sure if the results included in this thread for tested AVs are truly correct.

I repeated the test 25 times for Defender at default settings and the results were as follows:
12, 12, 12, 11, 10, 6, 9, 8, 7, 5, 7, 7, 7, 7, 3, 6, 7, 18, 17, 20, 16, 18, 20, 16, 16 ~ on average 11 vulnerables per 1 test.

So, it is possible that in a single test one could get the result 20 vulnerables and also 3 vulnerables.:sick:

To be sure, one should repeat the test several times for each AV.
 
Last edited:

miguelang611

Level 2
Thread author
Apr 13, 2020
99
I am not sure if the results included in this thread for tested AVs are truly correct.

I repeated the test 25 times for Defender at default settings and the results were as follows:
12, 12, 12, 11, 10, 6, 9, 8, 7, 5, 7, 7, 7, 7, 3, 6, 7, 18, 17, 20, 16, 18, 20, 16, 16 ~ on average 11 vulnerables per 1 test.

So, it is possible that in a single test one could get the result 20 vulnerables and also 3 vulnerables.:sick:

To be sure, one should repeat the test several times for each AV.
Wew, that is so much of a change!! Defender I just tested once, but the others there was like 10/15 days between my first test and when I did it properly for uploading it here and nothing really changed... But well, that's 2/3 times, not 10 hehe

By the way, thanks to the mod who edited my OP and added spoilers. It is so much better now!

See you!
 
  • Like
Reactions: Andy Ful

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top