I think this quote explains it:
As for why not everyone uses this (including Emsisoft), the number of FPs would be insane. All great having 100% detection, but your product will become extremely unusable and cause damage.
Emsisoft static detection - is it getting worse?
- Thread starter Malakke
- Start date
Please provide comments and solutions that are helpful to the author of this topic.
- Status
As for why not everyone uses this (including Emsisoft), the number of FPs would be insane. All great having 100% detection, but your product will become extremely unusable and cause damage.
- Feb 7, 2014
- 1,540
All I read here is facts put forward by Lucent Warrior & all he is getting is dodgy answers this does not help,especially coming form someone in who ( Fabian Wosar) I thought will have been keen to grab this to help make the product better, I'm a long time user and i can say this for a fact Emsisoft is clearly struggling, now I can see why.
- Sep 14, 2014
- 1,041
As on of the testers, I'd like to add my 2 cents.
I don't do these tests for money or similar (even not for "fame" if you want).
I do it because I can and I'm into testing samples. Do I do it right? Of course not. Many times I can see that sample is VM aware and doesn't do anything or terminates quickly. This doesn't mean that AV product missed it, but it did not detect it either (of course BB needs more time to check the behavior etc etc...).
While we can add all the stuff (Regshot and what not) to the testing tools, I think it comes down to the simplicity of these tests. Including all these logs etc would definitely improve the "missed" portion of the tests, but in the end it's all about our free choice to run these samples. More tools increases testing time and I don't really want to spend that much of my free time running a couple of samples out of 300 000+.
But we'll see. Perhaps we can improve on that
Of course there are endless IFs and one could say "Then don't do it" (malware testing). As Fabian said, doing it correctly is difficult and we're not trying to be next AV-T or AVC
I don't do these tests for money or similar (even not for "fame" if you want).
I do it because I can and I'm into testing samples. Do I do it right? Of course not. Many times I can see that sample is VM aware and doesn't do anything or terminates quickly. This doesn't mean that AV product missed it, but it did not detect it either (of course BB needs more time to check the behavior etc etc...).
While we can add all the stuff (Regshot and what not) to the testing tools, I think it comes down to the simplicity of these tests. Including all these logs etc would definitely improve the "missed" portion of the tests, but in the end it's all about our free choice to run these samples. More tools increases testing time and I don't really want to spend that much of my free time running a couple of samples out of 300 000+.
But we'll see. Perhaps we can improve on that
Of course there are endless IFs and one could say "Then don't do it" (malware testing). As Fabian said, doing it correctly is difficult and we're not trying to be next AV-T or AVC
Last edited:
You can't say it wouldn't detect it, without testing it on a hardened VM. If the sample doesn't show any malicious behaviour as most VM-aware malware does then the behaviour-based detection won't happen. Anything which does uses something different to behaviour-based detection (cloud, for example).
- Jun 29, 2014
- 260
Feel free to point out the facts you feel I haven't addressed or have only addressed in a dodgy manner.
- Nov 19, 2014
- 2,350
Let's not scare away one of the few developers that care to take part at conversations. Sure we can't all agree but let's agree to disagree.
- Jun 29, 2014
- 260
Oh don't worry, you don't get rid of me that easily. In general I don't mind arguing with people. As long as people don't start attempting to explain to me how our products work or how systems work, that they never had access to, we are fine
If the Developer can only come in to condemn and not spread some thanks for efforts by the Volunteers because the product does not appear to be fairing well lately in tests, instead of realizing that what we are all viewing lately is not exactly the products fault so much as the signatures provided from Bitdefender not being up to par as much as normal, then of what use is this. All products have good and bad days with signatures, hence why i chose to employ Deny/Default security and not rely on those.
Just knowing he enjoys watching his product shine in youtuber tests with old samples over viewing a test where the whole product is tested with fresher lower detection samples tells me plenty.
Im quite tired of hearing how these "Money in hand" organizations are the professionals, and all of us are basically idiots, and that from the view points here, we should rip down the Hub and close MT as we know nothing.
Being endorsed by Vendors for the wrong reasons does not make them right.
Last edited by a moderator:
+1
Absolutely agree.As on of the testers, I'd like to add my 2 cents.
I don't do these tests for money or similar (even not for "fame" if you want).
I do it because I can and I'm into testing samples. Do I do it right? Of course not. Many times I can see that sample is VM aware and doesn't do anything or terminates quickly. This doesn't mean that AV product missed it, but it did not detect it either (of course BB needs more time to check the behavior etc etc...).
While we can add all the stuff (Regshot and what not) to the testing tools, I think it comes down to the simplicity of these tests. Including all these logs etc would definitely improve the "missed" portion of the tests, but in the end it's all about our free choice to run these samples. More tools increases testing time and I don't really want to spend that much of my free time running a couple of samples out of 300 000+.
But we'll see. Perhaps we can improve on that
Of course there are endless IFs and one could say "Then don't do it" (malware testing). As Fabian said, doing it correctly is difficult and we're not trying to be next AV-T or AVC
I read trough all those replies, and when I was first notified that there are replies of you, @Fabian Wosar, I was thinking about constructive input on how to improve the testing.
But all that I can read is bashing on people trying to help an incomplete software getting more complete. This is pretty shocking if you ask me. If Emsisoft would have an infected rating once in a blue moon, I'd understand the swagger. But with an infection at least once a week, with about <100 samples being processed in 7 days, we're far from being allowed to show a conduct like that.
Testing the way @Fabian Wosar suggested I'd be delighted to do that for money, working full time at Emsisoft.
Unfortunately, I already have a full time job, friends, a tough study, and many other interests than sitting in front of my PC the whole day.
I spent enough time trying to perform those tests as perfect as possible, with on some days results in stress, and unlike as from your customers which should be most important as they pay your wages, Emsisoft (better Fabian Wosar) has nothing better to do than telling me that everything is crap. Like as @Lord Ami, I just test for fun, not for money, not for reputation, but trying to improve the product I use.
For plenty of details, experts might want to consult the HybridAnalysis links which are mandatory when posting new malware packs.
I will happily switch to another product which offers 100% protection out of the box (for example Kaspersky as to be seen in the Malware Vault, backed by a experienced tester), with maybe some false positives (yes, I am aware of the Zemana case), leave the Malware Testing part of the place with the words "tips" and "Malware" in it's address, but better safe than sorry. I don't care about one more message popping up as long as the software manages to protect my files by a bulletproof BB / HIPS (or picks it up with signatures). And I think most of the users here will agree.
We have plenty of examples in the Malware Vault which caused the system to get infected, which means the samples were working as they were supposed to.
As our tests cause lot's of trouble, I will give it into the hands of the SuperMods and the Admins whether and how we shall continue with Emsisoft testing.
Me is not willing to extend the tests to the "professional" standard you expect @Fabian Wosar, I'll rather drop both the product and my badges I earned for continious testing of Qihoo 360 Total Security and now Emsisoft Internet Security 11, until now without any complaints.
My time is too important, as I'd need to try to earn the respect of an AV vendor which obviously does not respect the work of it's users at all.
- Jul 28, 2016
- 950
wow those are some hefty words right there , but you have hit the nail right on the head! my opinion is that devs should be proud and glad to get help from amateur testers , to furthermore devellop the product and not to have an arrogant posture against people who spent their free time / willingness to test products out. when I read this all I am changing my mind , whether to buy emsisoft is when my norton subscription runs out. the basic of great product is its team of devellopers , withou cocky attitude.
- Jan 29, 2013
- 221
Shooting himself on the foot...
It was never my plan to forge users opinions, nor to reply more hefty than needed, I'm just not as good as finding the right words as @Lucent Warrior tried to but obviously ran against a wall.
I was just surprised by the way the developer reacted.
The product itself is good to go, and there will always be security gaps in any software, which can be easily shown off with the low level of testing we perform, but unfortunately I'm no programmer. To help the developer fix that, I try to offer as much information as possible, still trying to not make the test look bloated and easy to understand for novice users trying to get resources for being able to decide for a product.
To sum it up, it's really nice to see that we testers have the backup of the people we do the tests for - the customers.
Oh don't worry, you don't get rid of me that easily. In general I don't mind arguing with people. As long as people don't start attempting to explain to me how our products work or how systems work, that they never had access to, we are fine
Pretty much sums up the whole argument. The last 2 pages can be summed up to this: people telling Antivirus and developers how to do their jobs and how their product works, though they do not even work in the field, nor do they know how the product really works.
Also, the most ridiculous argument I've seen in this thread so far: I'm willing to test products the right way if the company behinds it pays me for it. But because they aren't, I'll just keep testing them using unreliable and random methods and if it fails to detect a sample, blame the product for it. Tell me, how is that helping the customer (and the company) exactly?
I can say that the relationship between AV devs and those who make malware testing (us) are usually cordial, although occasionally, an AV can show some problem, and so some tension can generate.
Our hope is not that devs need to accept our "modus operandi" but simply we do malware testing in the best possible way and, sure, it is always possible to improve.
AVs had a deep change during the last few years and now many products use some methods to detect and block the malware. The identification of potential harmful components through the use of signature is important, but the exponential increase in the spread of malicious software such as variants of other threats or the delivery of malware developed in a few samples, and not well known, makes the approach, based on the antivirus definitions, will not be very effective, then tests based just on antivirus signatures is not fully representative of how things go in the real world.
So we have tried to improve the HUB whereas vendors use also methodologies of detection based on behavioral analysis: in this way it is possible to detect the presence of a malicious component in the strength of the actions that it is trying to put in place.
The many ways with which a system can be attacked and possibly infected, makes extremely complex our work of AV testing. The different methods of attack involve the adoption of different defenses, each of which must be thoroughly tested to reach a final judgment objective.
Tests based on virus signatures can be completed in five minutes...and I assure you that in my case, a single complete test requires two hours!
There are, however, needs about malware samples as they have to be the most recent.
Of course, our tester perfectly know where and how to obtain fresh samples for a more reliable result as possible.
That said, the constructive criticism doesn't destroy a project, it allows you to understand any mistakes for a better work. Destructive criticism, however, affects the basis of the work and you can think that you are unprepared. Unsuitable. Incapable.
Behind every problem there is a challenge to win...highlighting only the problems, it's wrong for sure.
Our hope is not that devs need to accept our "modus operandi" but simply we do malware testing in the best possible way and, sure, it is always possible to improve.
AVs had a deep change during the last few years and now many products use some methods to detect and block the malware. The identification of potential harmful components through the use of signature is important, but the exponential increase in the spread of malicious software such as variants of other threats or the delivery of malware developed in a few samples, and not well known, makes the approach, based on the antivirus definitions, will not be very effective, then tests based just on antivirus signatures is not fully representative of how things go in the real world.
So we have tried to improve the HUB whereas vendors use also methodologies of detection based on behavioral analysis: in this way it is possible to detect the presence of a malicious component in the strength of the actions that it is trying to put in place.
The many ways with which a system can be attacked and possibly infected, makes extremely complex our work of AV testing. The different methods of attack involve the adoption of different defenses, each of which must be thoroughly tested to reach a final judgment objective.
Tests based on virus signatures can be completed in five minutes...and I assure you that in my case, a single complete test requires two hours!
There are, however, needs about malware samples as they have to be the most recent.
Of course, our tester perfectly know where and how to obtain fresh samples for a more reliable result as possible.
That said, the constructive criticism doesn't destroy a project, it allows you to understand any mistakes for a better work. Destructive criticism, however, affects the basis of the work and you can think that you are unprepared. Unsuitable. Incapable.
Behind every problem there is a challenge to win...highlighting only the problems, it's wrong for sure.
- Jan 29, 2013
- 221
Nobody has blamed Emsisoft! Quite the opposite... Usually has been praised as a great antimalware solution.
Are we reading the same thread? I can already see 2-3 users saying they won't recommend Emsisoft anymore, drop the product, etc.
- Jan 29, 2013
- 221
Only one user and certainly i understand his reaction. Things could be said in many ways... And sorry.. obviously english is not my main language
Interesting, I did not see one person telling Emsisoft how to do their job in this thread, i did how ever see many shocked by this developers arrogance and attitude. I also seen a developer come in here and bash many volunteers hard work, and basically state the forum was full of amateurs that know nothing, this includes yourself. I also seen a couple point out in the forum that BD's signatures were their concern and not the whole product, no Emsisoft bashing took place here. I did see how ever a Vendor, that rarely visits this forum, or at least does not post often helping users here, do nothing but degrade users here.
I did not so much hit a wall, as i realized this tennis tournament was going to either not end well, or just drag on, and all points have already been discussed whether absorbed or not.
I for one, have respect for this forum and members, and especially those volunteering to do these tests, as most users will never know or see how the product reacts to threats, never see what they are paying for, as well as samples being submitted to Vendors and or bugs found when doing this testing, is it really too much to just say Thank you for helping?
I would also like to point out, there is more to being a professional then just having knowledge of the product, how you carry yourself also reflects on this, and right now, im not seeing a professional representing the product, but an arrogant/rude man.
- May 14, 2016
- 1,597
- Jul 28, 2016
- 950
the peace is here in this thread , the love ( and respect )a representative of this company has for MT av - testers is sub zero. a healthy coversation is good. it's a shame to see that our fellow member av testers do not get the respect they deserve....there are other companies who have a deep respect for their ( beta ) testers and do apreciate each and every input , thats the way it is supossed to be.unfortunately not every company is the same. I had discusssions with Jasdev Dhaliwal , a high representative of AVG and I did get involved into a same situation like the one we have at hand here. he defended his point of view and claimed he and the company he represented was right , but in fact he was not. sometimes it courage to accept mistakes , it's character quality not everyone has.
Last edited:
- Status
Similar threads
