Serious Discussion Turtle‘s Enhanced Realworld Tests ( updated )

Kongo

Level 36
Verified
Top Poster
Well-known
Feb 25, 2017
2,562
If you are too sensitive about what other people say, then the answer on your comment is yes.
It has nothing to do with being sensitive. You're calling the test a waste of time and time to read. For me the only thing that seems like a waste of time, was you typing that comment in the first place. Absolutely no value for anyone.
 

Shadowra

Level 34
Verified
Top Poster
Content Creator
Malware Tester
Well-known
Sep 2, 2021
2,397
Nothing disrespectful at all, just some friendly advise. Such a waste of time for them to do this and all is wrong. So thats why i suggest, learn from @Shadowra

I see that I've been mentioned, so I'll respond and clarify this.

Firstly, I do not endorse any disdain towards a fellow tester. There are no courses on anti-malware testing; everyone has their own method. It is true that there may be a different or alternative approach, but the intention is the same.

Next: I do not have a diploma for testing, and I don't have to teach anyone. I can provide advice, but I do not give orders. I appreciate people recognizing my work, but I also expect respect for my colleagues (except for a few, but that's my personal matter).

I believe the goal is to highlight what's wrong with @ShenguiTurmi not to overwhelm them. While mixing AV Home, Endpoint, and the rest may not be good, it should be acknowledged that he and his participants have invested a lot of time. So, the least we can do is recognize the time spent. Personally, a video takes me 3 to 4 hours in terms of preparation (URLs, sample packs, recording, cleaning, editing).
 
F

ForgottenSeer 100397

@ShenguiTurmi, Your tests are valid. Some products have default settings, while others have custom settings. When viewed individually, they serve as good product tests. However, the issue lies in the language (not English) and the graphical representation of the results. The graphical representation may give the impression of a comparative test, which can be misleading. I understand that this was not your intention, as you mentioned the specific products and configurations in your post. I suggest posting separate tests for home and business products, considering the settings (default or custom). Keep up the good work!
 

Trident

Level 30
Verified
Top Poster
Well-known
Feb 7, 2023
1,960
I think we should all learn to appreciate the work people are doing pro-bono, even when it doesn’t necessarily tickle our fancy. And then we should all learn and move on. Nobody gets it perfect 100% of time. The test albeit not great for comparison, is still beneficial to see how products react in different circumstances.
 
F

ForgottenSeer 103564

I think we should all learn to appreciate the work people are doing pro-bono, even when it doesn’t necessarily tickle our fancy. And then we should all learn and move on. Nobody gets it perfect 100% of time. The test albeit not great for comparison, is still beneficial to see how products react in different circumstances.
This is not to target anyone or cause strife but a legitimate question.

If the testing methodology is flawed and reflects upon the product in a poor light when it's not accurate, how does this benefit anyone, user or company?

This question does not apply to the OP as much as the general statement that any testing is ok.
 

Anthony Qian

Level 10
Verified
Well-known
Apr 17, 2021
450
The test used APT frameworks for a more realistic simulation, not just basic scans or execution tests. That’s valuable info.

As for the custom settings on certain enterprise products like Deep Instinct, real-world companies customize their settings too. So, it’s not unfair or meaningless. All of the home products are tested with default settings so it’s comparable.

Criticizing the whole test for this small detail misses the bigger picture.
 

Trident

Level 30
Verified
Top Poster
Well-known
Feb 7, 2023
1,960
If the testing methodology is flawed and reflects upon the product in a poor light when it's not accurate, how does this benefit anyone, user or company?
The light is not necessarily inaccurate. The developer provides these settings and products, and it is possible that a user may be running these. Products have not been damaged or in any way tampered with. Users are responsible for doing their own research, not all information has to be pre-chewed and put down their throat.

For the next tests @ShenguiTurmi will do better.
 
F

ForgottenSeer 103564

The light is not necessarily inaccurate. The developer provides these settings and products, and it is possible that a user may be running these. Products have not been damaged or in any way tampered with. Users are responsible for doing their own research, not all information has to be pre-chewed and put down their throat.

For the next tests @ShenguiTurmi will do better.
As stated above, this was not to reflect upon the OP of this thread and his test but just to point out that not all testers methods should be praised as their methods can cause more damage than produce good. This is to not sway the OP from further improvements and tests but to remind all that users learn from these and it reflects upon companies, and accuracy needs to be adjusted with this in mind. No offense was meant to anyone with these mentions.
 

Trident

Level 30
Verified
Top Poster
Well-known
Feb 7, 2023
1,960
As stated above, this was not to reflect upon the OP of this thread and his test but just to point out that not all testers methods should be praised as their methods can cause more damage than produce good.
I can’t agree with this statement. Providing any sort of information that has not been manipulated purposefully can’t and won’t cause damage to anyone. Again, it is the forum reader’s responsibility to conduct their research, try different products and discover what suits their budget and needs. There is a disclaimer on top as well. People that have come across MalwareTips and this thread are already knowledgeable enough to draw the right conclusions. The average Joe who thinks trojan horse is one virus and also the scariest won’t even reach this or any other test.
They would be fine with their OEM Norton and McAfee.
 

ShenguiTurmi

Level 3
Thread author
Well-known
Feb 28, 2023
126
This "test" totally not representable at all. Mix of Hard / Normal settings..... please learn how to test by contacting @Shadowra , your test is a waste of time ( and time to read )
Next please.....
I don't mind repeating what I said before out of courtesy:
For consumer security software, keep the default settings except for Trend Micro, which has automatic highly sensitive toggling turned on by default, which is not realistic. Looking at the results, even if it's not turned off it doesn't make a difference as it still blocked all 5 binaries in the test on the non-highly sensitive setting.

@ShenguiTurmi, Your tests are valid. Some products have default settings, while others have custom settings. When viewed individually, they serve as good product tests. However, the issue lies in the language (not English) and the graphical representation of the results. The graphical representation may give the impression of a comparative test, which can be misleading. I understand that this was not your intention, as you mentioned the specific products and configurations in your post. I suggest posting separate tests for home and business products, considering the settings (default or custom). Keep up the good work!
Next time, I will at least divide the Enterprise and Customer product lines into two diagrams and mark their settings directly on the diagram. As for the video, next time I will find friends living in Canada to help me post it on YouTube. Thank you for your suggestions.

I think we should all learn to appreciate the work people are doing pro-bono, even when it doesn’t necessarily tickle our fancy. And then we should all learn and move on. Nobody gets it perfect 100% of time. The test albeit not great for comparison, is still beneficial to see how products react in different circumstances.
Thx a lot.
 

Jengo

Level 9
Well-known
Nov 9, 2022
417
I don't mind repeating what I said before out of courtesy:
For consumer security software, keep the default settings except for Trend Micro, which has automatic highly sensitive toggling turned on by default, which is not realistic. Looking at the results, even if it's not turned off it doesn't make a difference as it still blocked all 5 binaries in the test on the non-highly sensitive setting.


Next time, I will at least divide the Enterprise and Customer product lines into two diagrams and mark their settings directly on the diagram. As for the video, next time I will find friends living in Canada to help me post it on YouTube. Thank you for your suggestions.


Thx a lot.
If you put Norton to all Aggressive, bet for sure all will be blocked ;-) keep it up !
 

Jengo

Level 9
Well-known
Nov 9, 2022
417
May be, but I won't make any changes to the settings of consumer grade products.
I never expected that the enterprise grade configuration (although it is our daily usage setting) would cause so much controversy.
Enterprise is NOT Norton, thats Symantec ( Broadcom ). Norton is LifeLock since years ( Gen )


 

Kongo

Level 36
Verified
Top Poster
Well-known
Feb 25, 2017
2,562
not sure how aggressive my Di settings are, but ref non-malicious, I've only had 1 false positive in past 160 days, and Di was not wrong in blocking it.
I'm using the most aggressive settings possible since day one, and had quite a lot of false positives. I added all the exclusions necessary to prevent false alerts. Now it's running perfectly fine, as I barely install any new software.
 

ShenguiTurmi

Level 3
Thread author
Well-known
Feb 28, 2023
126
Enterprise is NOT Norton, thats Symantec ( Broadcom ). Norton is LifeLock since years ( Gen )


Yes, I know. But I don't have license for Symantec, so I didn't test it. Although it does not require activation authorization locally like FSCS/Trellix, I believe there may be some differences in whether to use the console (such as FSCS unable to set firewall and rollback policies without console).
Conversely, I don't have the personal version of Bitdefender (their reseller in China only sell the enterprise version), nor can I trial it (Bitdefender's official website blocked Chinese Mainland IPs), so I only tested their enterprise version.
 

Shadowra

Level 34
Verified
Top Poster
Content Creator
Malware Tester
Well-known
Sep 2, 2021
2,397
I'm using the most aggressive settings possible since day one, and had quite a lot of false positives. I added all the exclusions necessary to prevent false alerts. Now it's running perfectly fine, as I barely install any new software.

This is what I had on my personal PC (it had detected a game like PUA...)
On my crash test PC, no problems with DeepInstinct
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top