Status
Not open for further replies.

Dima007

Level 22
Verified
Redditor runs his own tests to find that almost all top Anti-virus softwares fail miserably
We install antivirus software on our PCs and laptops based on its reputation. For reputation we normally trust the reviews and ratings given for that particular antivirus software.

Redditor, man_on_the_train went a little further and devised his own tests to find out how the Antivirus softwares fare. He tested the most popular security software using Matousec’s SSTS, CLT and his own keylogger software.

Man_on_the_train found that almost all top names in antivirus softwares failed in his test save SpyShelter Firewall 10.0 which passed all his tests with a score of 99%. Incidentally, SpyShelter is not a full blown antivirus. Its more of a Firewall which doubles up as AV. Man_on_the_train found that only SpyShelter passed Matousec’s SSTS, CLT and his own keylogger software tests.

“SpyShelter does excellent job in protecting itself from being killed off by malware. It does detect attempts of executing malicious code through Task Scheduler.” Man_on_the_train adds, “BITStest has proven that advanced malware be unable to do anything if user decides to block the action.”



As you can see from the image about almost all of the top Antivirus softwares failed man_on_the_train’s self devised tests. Out of the top AV software, Avira Free Antivirus, McAfee LiveSafe Internet Security and Avast Premier failed to score any grades in Man_on_the_train’s books.

Comodo Internet Security Pro managed to pass the Kill5 Test, get a CLT score of 340/340 and pass Zero-day malware test. Man_on_the_train gave it highest score of 60 % after SpyShelter.

ESET Smart Security, Kaspersky Total Security 16, Zone Alarm Extreme Security managed to eke out 20 % score on man_on_the_train’s benchmarks. While BitDefender Total Security 2015 managed a 4% score and Norton Security managed a paltry 2%. ESET and Kaspersky managed to pass the Kill5 Test and Zone Alarm managed to pass the Zero-Day malware test.

Here are the conclusions given by Man_on_the_train for each Antivirus software.

  • Comodo Internet Security : Comodo Internet Security Pro will get easily killed off by malware if you do not turn on your HIPS manually. I tested it initially with HIPS on, and it passed kill5 test. Then I turned off HIPS module (default setting) and it got killed off easily. It does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with Comodo installed.
  • ESET Smart Security : ESET Smart Security has passed kill5 test. This is one of the most sophisticated kill methods, and ESET defends itself well. On the other hand, it does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with ESET installed.
  • Kaspersky Total Security : Kaspersky Total Security does protect itself well against sophisticated process killing methods, however it does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with Kaspersky Total Security installed.
  • Zone Alarm Extreme Security : ZoneAlarm has failed all SSTS64 tests. It will get easily killed off by malware. It has failed kill5 test so it can be easily killed. It does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with ZoneAlarm Extreme installed.
  • BitDefender Total Security 2015 : Bitdefender Total Security has failed all SSTS64 tests. It will get easily killed off by malware. It has failed kill5 test so it can be easily killed off by malware. It does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with Bitdefender installed.
  • Norton Security : Norton Security has failed kill5 test. This allows any sort of virus to effectively shut down Norton Security. It does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with Norton Security installed.
  • Avira Free Antivirus : First of all, Avira processes can be easily killed by malware. This allows any sort of virus to effectively shut down Avira. Avira does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with Avira Free Antivirus installed.
  • McAfee LiveSafe Internet Security : McAfee Security Center has failed kill5 test. This allows any sort of virus to effectively shut down McAfee LifeSafe Internet Security.It does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with McAfee installed.
  • Avast Premier : Premier has failed kill5 test. This allows any sort of virus to effectively shut down Avast! Premier. It does not protect against executing malicious code through Task Scheduler. On top of that, BITStest has proven that advanced malware will update itself without any issues with Avast! installed.
Man_on_the_train has also posted elaborate videos of his tests which can be accessed here.

Next time you need to buy a anti virus solution for your PC or Laptop, you know which one to publish. Kindly upvote man_on_the_train for his efforts on Reddit here.
 

jamescv7

Level 61
Verified
Trusted
Typically antivirus firms primarily focus to detect and
prevent viruses/malware in such very tradition even though many techniques are covered that must be optimize. It relies on HIPS which should covered any critical situation however its beyond user interactions.

Tweaks make it stronger than default as always, bundled components like Default deny to prevent an unknown application to execute or sandbox which will run immediately based on the behavior can covered that situation as an assurance its real time protection failed.
 
H

hjlbx

“Man_on_the_train adds, “BITStest has proven that advanced malware be unable to do anything if user decides to block the action.”

Read the above sentence. Read it again. Now think. What does it mean ? If you allow it to run on your system, then it can infect your system. I'll tell you in a word what it all means: Antiexecutable !

Hah,hah,hah...Hee,hee,hee...hooh,hooh,hooh...


NOTE: The utilities that the OP used are rated as Trusted by Comodo. Therefore, CIS will permit those utilities to run without restriction - and make changes to the system (there are limited exceptions). Few users completely understand that CIS protection applies for the most part only to Unrecognized files.

Bottom Line = the test is critically flawed. If OP had changed utility ratings from Trusted to Unrecognized, then CIS would have returned all alerts\blocks...

I suppose the OP's point is that AV softs at default settings do a poor job of protecting the system. In other words, AVs don't provide much protection because they are poorly configured with settings that are intended to "coddle" a novice - instead of default settings intended to provide high-level security. I agree... he does demonstrate that fact. He also points out that a typical user will be fully unaware of this fact. Here again, I totally agree...

On the other hand, what he does demonstrate can be interpreted by some as spreading FUD. Advanced users know that AV default settings are flawed protection-wise; they immediately go about tweaking their AV configuration. Moreover, Matousec\Standford (SSTS64) tests are always performed against AVs that have been configured with maximum settings. The entire premise of Matousec testing is that default settings are essentially worthless... !!

So, in short, it is my opinion that the OP is not spreading FUD. He's just demonstrating that default settings for virtually all AVs provide inadequate protections. Nothing new there... right ?

As an aside...

At default settings I am not surprised in the least to see Comodo not pass some tests. However, I would bet that if the OP had applied the Matousec CIS configuration, then it would have passed all tests.

Comodo can be configured by the user to provide lax through lock-down grade security. This flexibility is one of its real strengths.

As a Comodo user I am curious to know precisely what settings modifications were\are made by Matousec testers when configuring CIS. Knowing very little about what they do, I suspect that they use HIPS Paranoid Mode. Honestly, I have never bothered to look it up since I have full confidence in my current config.

On my system I have configured Comodo very similar to an AppGuard\NoVirusThanks Exe Radar Pro combo. If the OP tested it against my Comodo configuration, then none of the tests he performed would, nor could, even launch !!

The advantage to Zemana is that "out-of-the-box" it has maxxed-out settings.

I see how the AVs were tested... and the results. I remain unconcerned...
 
Last edited by a moderator:

XhenEd

Level 27
Verified
Trusted
Content Creator
I was listening to Malware Doctor on youtube, and he doesn't care about, nor rate antiviruses that focus on detection as it means nothing as the number of viruses in the wild is unknown. Going back to my Physics days:

In probability theory, the central limit theorem (CLT) states that, given certain conditions, the arithmetic mean of a sufficiently large number of iterates of independentrandom variables, each with a well-defined expected value and well-defined variance, will be approximately normally distributed, regardless of the underlying distribution.[1][2] That is, suppose that a sample is obtained containing a large number of observations, each observation being randomly generated in a way that does not depend on the values of the other observations, and that the arithmetic average of the observed values is computed. If this procedure is performed many times, the central limit theorem says that the computed values of the average will be distributed according to the normal distribution (commonly known as a "bell curve").

Quoted from: Central limit theorem - Wikipedia, the free encyclopedia
This page was last modified on 14 August 2015, at 08:54
Speak English, please... :p
I don't even understand a thing except the first sentence. :D
 

Enju

New Member
AV software can be killed by an infection?! Scandalous! :rolleyes: Sorry I just can't take such a test seriously as soon as somebody uses so called self-protection as a criteria... seems more like somebody from the SpyShelter team had a bit of spare time and wanted to show how awesome their software is against cherry picked testing scenarios.
 

Atlas147

Level 30
Verified
Content Creator
I'm sure if there are some modifications to the settings in each of the AV there would be a significant amount of improvement, this is why you cannot use default settings in all softwares. Don't be lazy!! Go into your settings and see what each component does and learn how to strengthen them.

As for Comodo, it's a really good product, however it's really annoying how it gives a pop up for everything you run, running a simple installer could take twice as long, if not longer because you have to accept all the prompts one by one. If you set it to accept all other prompts from this installer then what's the use of the prompts in the first place? Too much nagging for me.

Have never used spyshelter, but from the fact that I have not heard much about it since I have joined the forum it doesn't seem to be a very popular programme. From the comments seen from the other thread about BSOD compatibility issues when you pair AVs up with it, it seems that it's not a very good software.
 

Kuttz

Level 12
Verified
As a general rule no security software is fool proof. Here the the author claims SpyShelter scores 99%. In a different security scenario that SpyShelter would perform utter failure even worse than most of the security software he tested. Any security software has its strength and weakness and it seems that SpyShelter is specialised to protect in Task Scheduler and may be a mediocre in other security features. Not impressed his way testing.
 

done

Level 5
Verified
Thats a realistic test. most of the antivirus I've tested with @Malware1 packs failed. I never tested kaper or emsi.
Panda fail
MSE fail
Avast fail
ESET fail
forticlient fail
AVG fail
Avira fail
sandboxie failed
Timefreeze Paseed on the C but other partitions got infected\encrypted
360 IS did better than all of them in all tests.


Comodo Passed But it needed lots of space to revert changes
private fire wall passed but it is blocking alot of legit programs.

So at the end of the day Non of the product did 100%.
 
  • Like
Reactions: XhenEd and Kuttz
Status
Not open for further replies.