App Review Real World, Evasive Malware and Performance Test by Trident

[Trident]

Welcome to this test performed by me.

The test focuses on 4 key areas:
Real World Protection Test (most important)
Evasive Malware Test ( important)
Antivirus Performance Test (important)
Stability/User Experience (important)

Every product has an initial score of 100 points.

Failure deductions are as follows:
Real World Protection Test Failure -( -100 and no further testing performed).
Evasive Malware Test (Rat/Infostealer -40; Ransomware no exfiltration - 20; Ransomware with Exfiltration -50; Ransomware, no exfiltration and product offers backup -10)
Performance Test (Abnormal size on disk [>1gb -10]; Abnormal Memory Usage [most systems have 16 GB so above 2% of that or 320 MB is abnormal] -10; Abnormal CPU Usage on Idle -10)
Stability/User Experience Failure (Credit card required to download -1; Bombarding with notifications -5; Interface does not meet UI trends in the last 10 years -10; Interface is difficult to get around -7; Installer is online-based but does not install the most recent version -1; Product offers only minimal configuration -2; Updates are very slow -5)

All unedited tests can be seen in this playlist:

Some evidence for performance will be displayed there, other will be in a screenshot.
Edited Video called "Falling Castles: Antivirus Test" is coming soon.

ZoneAlarm Extreme Security NextGen by Check Point:
Real Word Protection Test: -0 (pass)
Evasive Malware Test (-40 for missed RATs and -20 for successful ransomware infection)
Performance (Abnormal Size on Disk -10; Abnormal Memory Usage -10)
User Experience (Bombarding with notifications -5)
Total = 15 points

F-Secure
Real World Protection Test -0 (pass)
Evasive Malware Test (-40 for missed RATs)
Performance (pass)
User Experience (pass)
Total = 60 points

Bitdefender Total Security:
Real World Protection Test -0 (pass)
Evasive Malware Test -0 (pass)
Performance (Abnormal Size on Disk -10, Abnormal Memory Usage -10)
User Experience (Installer is online based but does not install the most recent version -1; Updates are very slow -5)
Total = 74 points

Kaspersky Security Standard:
Kaspersky allows trial to be downloaded but uses tricks to make it more difficult for users not familiar with their website and tools.
Real World Protection Test -0 (pass)
Evasive Malware Test (-40 for missed RATs)
Performance (pass)
User Experience (Bombarding with notifications -5)
Total = 55 points

Norton 360 Advanced
Real World Protection Test -0 (pass)
Evasive Malware Test (-40 for missed RATs even though IPS blocks connection. A miss is a miss)
Performance Test (pass)
User Experience (Credit card required to download -1; Bombards with notifications -5; Installer is online based but does not install the most recent version -1)
Total = 53 Points

Avast Free (gets 10 extra points because it's free; did not test Kaspersky Free as it has no Application Control)
Real World Protection Test -0 (pass)
Evasive Malware Test (-40 for missed RATs)
Performance Test (Abnormal size on disk -10)
User Experience (pass)
Total = 60 points

Eset Smart Security Premium:
Real World Protection Test -0 (pass)
Evasive Malware (-40 for missed RATs)
Performance Test (pass)
User Experience Test (Bombards with notifications -5; Installer is online based but does not install the most recent version -1)
Total = 54

Worth mentioning:
Microsoft Edge did great job blocking malware links (I didn't turn off its protection because users have no reason to do so)
Microsoft Defender was disqualified even before testing as apparently some malware managed to damage it. It was just loading continuously.

Overall winners:
Champion: Bitdefender
Second best: F-secure; Avast
Third best: Kaspersky Standard; Eset Smart Security Premium (more expensive than Kaspersky)
To be avoided: ZoneAlarm

This concludes this year's testing.

I would like to apologise to viewers if I look confused in some of the videos (the mouse movement). I am a Mac user and all this malware, notifications, menus (one inside another) is extremely overwhelming.

Spoiler: Screenshots

[/SPOILER

 

[M4RT1NE2]

Bravo BD, poorly Norton....

 

[Jonny Quest]

No worries about the mouse movements. The videos I've watched so far have been easy to follow and track. Thank you, and well done

 

[a090]

Nice work, brother. Seems my boy F-Secure did alright. As a part-time Mac user myself, I can understand the frustration with all those erratic popups. Didn’t mess with my viewing pleasure though.

Keep it up; good attention to detail in the testing as well.

Edit: How do I give reputation points or whatever those green things are? New to this forum platform… looks like Xenforo but not sure. Anyway, this post of yours deserves more than just a like… @Trident

 

[Trident]

Nice work, brother. Seems my boy F-Secure did alright. As a part-time Mac user myself, I can understand the frustration with all those erratic popups. Didn’t mess with my viewing pleasure though.

Keep it up; good attention to detail in the testing as well.

Edit: How do I give reputation points or whatever those green things are? New to this forum platform… looks like Xenforo but not sure. Anyway, this post of yours deserves more than just a like… @Trident

Thanks. you need to hover over the "Like" and then other reactions appear.

 

[a090]

Thanks. you need to hover over the "Like" and then other reactions appear.

Wow… that was easy lol. Thanks, I was able to add rep to your post.

 

[Shadowra]

Too bad you didn't include Eset.
On the other hand Bitdefender impresses me and confirms the video I'm preparing on him

 

[M4RT1NE2]

Too bad you didn't include Eset.
On the other hand Bitdefender impresses me and confirms the video I'm preparing on him

I'm looking forward to this

 

[Trident]

Too bad you didn't include Eset.
On the other hand Bitdefender impresses me and confirms the video I'm preparing on him

Testing Eset now. Will edit the post. To be honest, I completely forgot they exist

 

[Trident]

I will edit the post and will include the results here as well.

Eset Smart Security Premium:
Real World Protection Test -0 (pass)
Evasive Malware (-40 for missed RATs)
Performance Test (pass)
User Experience Test (Bombards with notifications -5; Installer is online based but does not install the most recent version -1)
Total = 54

 

[Shadowra]

I will edit the post and will include the results here as well.

Eset Smart Security Premium:
Real World Protection Test -0 (pass)
Evasive Malware (-40 for missed RATs)
Performance Test (pass)
User Experience Test (Bombards with notifications -5; Installer is online based but does not install the most recent version -1)
Total = 54

So Bitdefender > Eset......

 

[Trident]

So Bitdefender > Eset......

According to the framework that I've set, yes. The Eset video is uploading as well. In previous videos I displayed evidence that the Java file is a pastebin-controlled RAT, but in this video I did not. This evidence can be seen in F-Secure and Avast videos.

 

[Trident]

A more in depth performance test may arrive later. Not everybody needs protection against evasive malware (though we can argue here). Some people would prefer a more performant solution. The next step would be to analyse the privacy policies.

 

[a090]

A more in depth performance test may arrive later. Not everybody needs protection against evasive malware (though we can argue here). Some people would prefer a more performant solution. The next step would be to analyse the privacy policies.

Now this is the content I subscribed for. Looking forward to this, please PM me when you begin working on this. Will be happy to help.

Now now, don’t start with me @Trident. We’re buddies now I’ll behave if you behave

 

[superleeds27]

User Experience Test

Deducting points for CC and notifications. Surely this is subjective, is it not?

So Eset/Kaspersky in theory are 60 too. Norton 59

 

[Trident]

User Experience Test

Deducting points for CC and notifications. Surely this is subjective, is it not?

So Eset/Kaspersky/Norton etc in theory are 60 too.

Notifications play a key role in software usage (as well as everything else you see). You don't see the behavioural blocking classifiers and the kernel driver code.
If you have a look on forums, complaints about notifications top. You are free to perform your own calculations.

 

[TuxTalk]

Bitdefender really good.

 

[Trident]

Bitdefender really good.

Apart from doing well in the visible part of the test, it removed malware that escaped Shadow Defender virtualisation by using BD Command Line Scanner. Nobody else detected that. Even Hitman Pro, Power Eraser and KVRT didn't.

 

[likeastar20]

what is the hash of the sample?

Apart from doing well in the visible part of the test, it removed malware that escaped Shadow Defender virtualisation by using BD Command Line Scanner. Nobody else detected that. Even Hitman Pro, Power Eraser and KVRT didn't.

 

[Trident]

what is the hash of the sample?

It wasn’t a sample, it was a scheduled task. I haven’t got it anymore, the system has been reset. I still have the malware archive and links though.