Battle Planned: Real-world Test of Trend Micro, ZoneAlarm, Eset and Webroot

[oldschool]

I would like to think of this forum as a enjoyable diversion, as we help each other out, learn and share from each other. So for me, it doesn't have to be university level

I believe most members feel this way, and guests realize this too. They can see users helping other users according to their knowledge and skill level.

As to testing methods, even guests can read in many places on MT to take all tests with a grain of salt. A test is just a snapshot in time. This one will be no different in this sense.

 

[lokamoka820]

I voted for Defaults because most users will use their products with default settings, so tweaking a security product lets say to the max security level will give simple users a wrong indicator about the product, unless if you do both tests, Defaults and Tweaked and put the results together for the same product, like testing Microsoft Defender with default settings, and when using configure defender max level.

 

[oldschool]

I voted for Defaults because most users will use their products with default settings, so tweaking a security product lets say to the max security level will give simple users a wrong indicator about the product

Same here.

Thanks @Trident for going to the trouble to setup these tests.

 

[mlnevese]

Let me clarify my position on tests. Even if an absolutely scientifically rigorous test was to prove without a shred of doubt that product X is the absolute best, most complete protection available against all kinds of threats and I was using product Y, I wouldn't just drop product Y because:

1) I already paid for the license
2) It's already installed, configured and running on my computers
3) The protection afforded by the best is probably less than 1% better than whatever I'm already using.


The only reason I'd switch immediately is:

1) a critical vulnerability was discovered on whatever I'm using, it's already being exploited and the vendor refuses to acknowledge or fix it.
2) some update/patch crashes or blocks something I need and there is no way to know when it will be fixed and no way to bypass it (actually had it happen with Bitdefender once)
3) my license is about to expire and I find a killing offer for some other product.

In other words, I have no favorites, I have used mostly anything that offers a good price in my country and always recommend to test any product before buying whenever possible.

 

[Trident]

I do not want anyone to switch to other security suites. Claims were made yesterday without any basis. This is the reason I am doing it. Not because I want everyone to uninstall what they are using and start using ZoneAlarm or anything else.

I was attacked and I was called something like charlatan that makes products look bad for profit.

These products are undoubtedly bad — everyone here knows it. But I was asked to create a video.

 

[oldschool]

Let me clarify my position on tests. Even if an absolutely scientifically rigorous test was to prove without a shred of doubt that product X is the absolute best, most complete protection available against all kinds of threats and I was using product Y, I wouldn't just drop product Y because:

1) I already paid for the license
2) It's already installed, configured and running on my computers
3) The protection afforded by the best is probably less than 1% better than whatever I'm already using.


The only reason I'd switch immediately is:

1) a critical vulnerability was discovered on whatever I'm using, it's already being exploited and the vendor refuses to acknowledge or fix it.
2) some update/patch crashes or blocks something I need and there is no way to know when it will be fixed and no way to bypass it (actually had it happen with Bitdefender once)
3) my license is about to expire and I find a killing offer for some other product.

And this is what MT readers will find in many posts and threads on the forum. Even the most casual reader can see and understand basic info like this, and not necessarily the most "professional", technical info.

 

[mlnevese]

I do not want anyone to switch to other security tests. Claims were made yesterday without any basis. This is the reason I am doing it. Not because I want everyone to uninstall what they are using and start using ZoneAlarm or anything else.

I was attacked and I was called something like charlatan that makes products look bad for profit.

Never thought you were but noticed my comment being taken out of context by others and decided to make clear what I think. As far as I'm concerned your test proposal is as close to a real world situation as a test can be without spending a lot of money to set up things

 

[Jonny Quest]

Let me clarify my position on tests. Even if an absolutely scientifically rigorous test was to prove without a shred of doubt that product X is the absolute best, most complete protection available against all kinds of threats and I was using product Y, I wouldn't just drop product Y because:

1) I already paid for the license
2) It's already installed, configured and running on my computers
3) The protection afforded by the best is probably less than 1% better than whatever I'm already using.


The only reason I'd switch immediately is:

1) a critical vulnerability was discovered on whatever I'm using, it's already being exploited and the vendor refuses to acknowledge or fix it.
2) some update/patch crashes or blocks something I need and there is no way to know when it will be fixed and no way to bypass it (actually had it happen with Bitdefender once)
3) my license is about to expire and I find a killing offer for some other product.

In other words, I have no favorites, I have used mostly anything that offers a good price in my country and always recommend to test any product before buying whenever possible.

Which I would also include stability (lack of glitches, issues) in your first three, which was also addressed with the #2 reason of why to switch (BD as well for me on two PC's).

 

[cruelsister]

They will execute from temp, desktop, downloads, whatever I decide to click (run or save). But the point is, the aggressive screening of downloads is not bypassed. Many products control false positives by using more aggressive screening of downloads, as opposed to local files.

What confuses me is that you have stated previously that executing a file from the Desktop is somehow not "real world", and should be differentiated from a file run from elsewhere (C:\Downloads, C:\Users\appdata\roaming, etc). In the same vein, how would a video authors decision to first do a custom scan then run undetected files from a malware pack in a C:\Malware directory be inferior?

Also, regarding the scanning abilities of various AM products- some will just be aware of the file ID, and if matched will detect; some will indeed look deeper (eg: if a link is received in Gmail and thereby stored somewhere in the browser directory in AppData, a good AM app can detect a malicious Powershell or Python string). But as such scans can be implemented from anywhere the malware resides, including malware packs, how is such testing "not real world" and somehow inferior?

Finally, the comment "Many products control false positives by using more aggressive screening of downloads, as opposed to local files." should be questioned. This is implying that an AM application utilizes some sort of Folder Priority, when the emphasis should actually be placed the potential system changes made by the execution the malicious file, either as a direct result of it or by the concurrent spawning of a LoLbin.

 

[Trident]

What confuses me is that you have stated previously that executing a file from the Desktop is somehow not "real world", and should be differentiated from a file run from elsewhere (C:\Downloads, C:\Users\appdata\roaming, etc). In the same vein, how would a video authors decision to first do a custom scan then run undetected files from a malware pack in a C:\Malware directory be inferior?

Webroot does use folder priority. That's how the product has been designed, I am not designing it. Other products of this sort are McAfee, Eset that uses emulation on downloaded files, Avast with their cyber capture that needs the MOTW. These products have additional systems that kick in only when the file is downloaded. The design decision of the vendor should be respected. Malware doesn't just come from the sky, It needs to be downloaded or saved from email (same thing). You cannot always place all products under one umbrella and test them the same way.
It is not even folder priority, it monitors the chain of events. Files created by browser are examined with a more aggressive rule set. AVs like Norton also take into account the website file came from.

To this point, I agree with Lynx. But I don't agree that users will start playing hide-and-seek or whack-a-mole, creating HIPS rules, tweaking heuristics. This is something that very small minority will do. Vast majority of users use Defender with no tweaks whatsoever.

But whatever, let's tweak them. They will still fail.

Finally, the comment "Many products control false positives by using more aggressive screening of downloads, as opposed to local files." should be questioned. This is implying that an AM application utilizes some sort of Folder Priority, when the emphasis should actually be placed the potential system changes made by the execution the malicious file, either as a direct result of it or by the concurrent spawning of a LoLbin.

This in the Webroot world is impossible, as it automatically whitelists and excludes absolutely all trusted binaries, which LOLBins are... it does not monitor them at all or scan them. This is why everyone would cast a doubt over the Webroot effectivness, which I did and then I was attacked.

 

[ForgottenSeer 114834]

To this point, I agree with Lynx. But I don't agree that users will start playing hide-and-seek or whack-a-mole, creating HIPS rules, tweaking heuristics. This is something that very small minority will do. Vast majority of users use Defender with no tweaks whatsoever.

But whatever, let's tweak them. They will still fail.

So what's all the fuss about if you have agreed with the initial assessment I placed forth?

I stated you should test the products abilities as designed. If it has advanced features that are disabled, you should enable them. Eset HIPS can be trained quickly in learning mode if you have very little in the test machine and then placed in smart mode which is kind of like interactive mode with minimal interaction.

These "tools" are designed to be used by those that understand the operating system and advanced settings of the product. Just because a handful of users here do not understand them or use them does not mean they are not utilized, most likely not by home users, but then again I know very few users here in the US using eset at home. They would look at the settings and be like " nope" of course they would understand it's not designed for users like them, ironic isn't it.

If you do not wish to use these features in a test, that's fine too, you just can't claim a product failed if you do not allow the product to utilize all its abilities to defend. This latter part is not rocket science.

 

[Trident]

If you do not wish to use these features in a test, that's fine too, you just can't claim a product failed if you do not allow the product to utilize all its abilities to defend. This latter part is not rocket science.

Eset will emulate these files. That should be enough. Eset charges premium for a product with emulation. Would be funny if it's not efficient enough. And then what about Phishing as well? Because the framework allows only 1 undetected page...
Will HIPS save Eset really?

 

[ForgottenSeer 114834]

Eset will emulate these files. That should be enough. Eset charges premium for a product with emulation. Would be funny if it's not efficient enough. And then what about Phishing as well? Because the framework allows only 1 undetected page...
Will HIPS save Eset really?

Only one way to find out really. If it fails then you have utilized the product as intended and no one would be able to claim otherwise as it was tested as designed to be used.

Simple really.

I'm not voting for any certain product and have found over the years actually all of them have something in common. I can use any of them and remain infection free.

I just have a problem with incomplete testing and users stating a product failed. That conclusion is misleading and needs to stop here, it's been going on way too long.

 

[Trident]

I just have a problem with incomplete testing and users stating a product failed. That conclusion is misleading and needs to stop here, it's been going on way too long.

The problem is that you proclaimed a product low-quality and its users incapable of evaluating properly. These were your words yesterday.

Now today, I am ready to test this product in defaults (I am already installing it the moment so tests are beginning).

You are persisting and insisting to test the solutions after various tweaks, when the product you are calling inferior doesn't need them.

So you see how it all becomes funny, don't you?

The "amazing" product that needs you to go through an admin course and the fully automated inferior one.

 

[oldschool]

Now today, I am ready to test this product in defaults (I am already installing it the moment so tests are beginning).

All right, we're looking forward to the results.

 

[ForgottenSeer 114834]

The problem is that you proclaimed a product low-quality and its users incapable of evaluating properly. These were your words yesterday.

Now today, I am ready to test this product in defaults (I am already installing it the moment so tests are beginning).

You are persisting and insisting to test the solutions after various tweaks, when the product you are calling inferior doesn't need them.

So you see how it all becomes funny, don't you?

The "amazing" product that needs you to go through an admin course and the fully automated inferior one.

I'm not sure where you got I called a product inferior from, as I did no such thing. I have not said a bad word about any product in this forum. I frown on product bashing. If you would kindly produce a screenshot of me bashing that product to refresh my memory, we can lay this crud to rest.

 

[Trident]

I'm not sure where you got I called a product inferior from, as I did no such thing. I have not said a bad word about any product in this forum. I frown on product bashing. If you would kindly produce a screenshot of me bashing that product to refresh my memory, we can lay this crud to rest.

 

[Divine_Barakah]

View attachment 284909

Well, let's be honest here. All he said was that ZA got hyped here which is true.

 

[ForgottenSeer 114834]

View attachment 284909

That's not bashing the product it's stating it wasn't tested true route of infection, there is a difference and you know this. The social phenomenon being that inaccurate testing took place. Not one word in there bashing the product.

 

[ForgottenSeer 114834]

Guys for God's sake let's move on. Let's wait for the tests and have a constructive discussion. This never-ending, back-and-forth conversation needs to end.

I don't know what's more ironic, that he took that position as he did, or the fact he's going to be up half the night trying to prove something from it.

Although if it provides test inf done correctly here in the forum I guess it will have been worth the obvious understanding impairment.