AV-Comparatives May 2017 test. Windows Defender did great!

[ttto]

taking these tests with a grain of salt, I'm happy with the results of windows defender. While some vendors go backwards, windows defender is getting better results. Also surpised Symantec rejoined after some time missing, fantastic results for Norton too.

 

[Parsh]

This clearly states that on Windows 10, nothing else is needed..... Or... It states
that the samples they used were clearly not zero day and possible had a few days age if not more, or you would not see these high scores,
which is, at best, misleading.

From what I've seen, MS Defender is getting good at catching new malware, only a few hours after the identification. It may not have a strong zero-day protection but the definitions of the new variants get quickly added to WD. Good that an ever-improving cloud component (as they have been advertising lately) is used to fortify their offering. The Win Pro users have an added advantage with advanced cloud modules.
I think it should be tested like just another AV product. Will be nice to see it tested at MH sometime

Avira results:

• March 100%
• April 99.6%
• May 99%

It seems to be a small decline...
Of course, we take these tests as a grain of salt, but thinking at the same time, I can think that malware is becoming more advanced and the Avira engine is old!
They should begin to think that it is time to rewrite a good part of their AV, especially the BB side, instead to add unnecessary crap functions.

Exactly! No one knows what they're upto, but it certainly looks like they very much need to improve (or implement rather?) a strong BB mechanism over simply relying on rules of behavior (sensors as they call) and then querying their cloud db with the concerned file info. Though this method may not be very different from some other AVs, this does not appear to be very effective in its case with new samples either. How about strengthening their offline behavioral/HIPS protection?
The differences in free and paid are obvious with some tests, though they claim that there are none with regards to their implementation.

 

[S3cur1ty 3nthu5145t]

MS Defender is getting good at catching new malware, only a few hours after the identification.

The samples you are using in the Malware Hub are never truly "FUD" zero day. They generally have a detection already of a few engines up to 25 at Virus Total, this is still low detection rate as not all engines are detecting them, but are in the process. What I mean by this is, once one of those packs are dropped into the Hub, wait about 24 hours then test it, and compare the engines that did detect it when the pack was first dropped until then, you will see many more engines now detecting it. What's not being taken into account, was how long that sample was in the Wild before being discovered and uploaded to one of the sites your grabbing samples from.

It may not have a strong zero-day protection

My point exactly. None of the Raw AV's have strong zero day protection. Matter of fact none of the "RAW Av's" can detect true zero day malware by signature, as the signature will not have been created for it yet, that is why most of them now have other modules like behavior monitoring ect.
Static testing of signatures is irrelevant because of this.
To see a sample pack tested and the product achieve 99.5% tells you these were not fresh samples.

It is misleading as I stated.

 

[Parsh]

The samples you are using in the Malware Hub are never truly "FUD" zero day. They generally have a detection already of a few engines up to 25 at Virus Total, this is still low detection rate as not all engines are detecting them, but are in the process. What I mean by this is, once one of those packs are dropped into the Hub, wait about 24 hours then test it, and compare the engines that did detect it when the pack was first dropped until then, you will see many more engines now detecting it. What's not being taken into account, was how long that sample was in the Wild before being discovered and uploaded to one of the sites your grabbing samples from.

Most of the samples tested here are not FUDs, yeah. That's why I used the words "after identification".
You never know how much time the malware has been in the wild. Even after some days, there can be close to 'NIL' detections on VT. However, once detections are being made, Defender usually catches up well and that's not seen with ALL AVs out there.

 

[mekelek]

this test actually seems realistic if we compare it to our own tests.
that 37 false positives on F-Secure, DeepGuard is a tough SOB
sadly ESET has been going downhill the past few months, barely detects anything without signatures. they need to develop new modules, HIPS is not doing it.

fairly surprised about the Bitdefender and Emsisoft results tho.

 

[Arin]

woww.. avast doing great job.

 

[cruelsister]

I've mocked Avast shamelessly in the past, but since the beginning of this year (with the melding of AVG) it is really getting quite strong. Although I am not fond of the traditional AV in general, in this class Avast is rising quickly to the top.

 

[Razza]

sadly ESET has been going downhill the past few months, barely detects anything without signatures. they need to develop new modules, HIPS is not doing it.

That the issue with Eset they not got a behaviour blocker in there product, unlike other vendors that been developing and improving there behaviour blockers in the last few years which can sometime block the malware without signatures but with Eset if the malware is not getting picked up by signatures your pc will most likely get infected.

 

[S3cur1ty 3nthu5145t]

That the issue with Eset they not got a behaviour blocker in there product, unlike other vendors that been developing and improving there behaviour blockers in the last few years which can sometime block the malware without signatures but with Eset if the malware is not getting picked up by signatures your pc will most likely get infected.

I think maybe you should get to know the product before you place statements like this. Eset is a full suite with HIPS. It does not rely on signatures only. Matter of fact, most users in this forum would be in serious trouble should they actually activate the Advanced Portions of Eset, by placing all of it in Interactive Mode, I wonder how many of you would be able to keep up with it.

 

[Janl1992l]

That the issue with Eset they not got a behaviour blocker in there product, unlike other vendors that been developing and improving there behaviour blockers in the last few years which can sometime block the malware without signatures but with Eset if the malware is not getting picked up by signatures your pc will most likely get infected.

That is true in default settings. but u can change hips so that u get popup massages for unknown processes etc. Eset need some tweaks for full protection, like comodo does.

 

[Fritz]

this test actually seems realistic if we compare it to our own tests.
that 37 false positives on F-Secure, DeepGuard is a tough SOB
sadly ESET has been going downhill the past few months, barely detects anything without signatures. they need to develop new modules, HIPS is not doing it.

fairly surprised about the Bitdefender and Emsisoft results tho.

Same here @mekelek. ESET scoring A for effort has been clearly visible in the MH as of late, but an entire year with F-Secure on 5 machines hasn't shown that many false positives. And I'm running all kinds of uncommon niche software.

 

[ravi prakash saini]

as per my personal testing I find the result about emsisoft and defender reliable.
do not known what is the problem with emsisoft it doesn't seem to me the same product it used to be two months ago.

 

[mekelek]

as per my personal testing I find the result about emsisoft and defender reliable.
do not known what is the problem with emsisoft it doesn't seem to me the same product it used to be two months ago.

well the results are not bad, Emsisoft asks you what to do and has most of the times the right choice recommended to you.

 

[Parsh]

as per my personal testing I find the result about emsisoft and defender reliable.
do not known what is the problem with emsisoft it doesn't seem to me the same product it used to be two months ago.

The Test results of Emsisoft have been more or less similar - missing a very few samples and some samples being user dependent. The latter can be improvised by setting the decisions to 'quarantine' or probably 'use recommended action'.
Even in case of many "yellow alerts" (against the red ones), the recommendation is usually "Quarantine" depending on the class of the suspicious activity detected.

I think comparing the current performance of a product with one - a few months back, just on the basis of testing on a few hundreds of random samples ain't justified.

 

[S3cur1ty 3nthu5145t]

Over the year we evaluate several tens of thousands malicious URLs. Unfortunately, many of these have to be discarded for various reasons. We remove duplicates such as the same malware hosted on different domains or IP addresses, sites already tested, “grey” or non-malicious sites/files, and malware/sites disappearing during the test. Many malicious URLs carrying exploits were not able to compromise the chosen system/applications because of the patch level. This means that the vulnerabilities in the third-party applications on the system were already patched and the exploits could therefore not deliver their malicious payload. Users should be aware that by always keeping their system and third-party applications up-to-date/patched, they can dramatically decrease the risk posed by exploits.

The results are based on the test set of 398 live test cases (malicious URLs found in the field), consisting of working exploits (i.e. drive-by downloads) and URLs pointing directly to malware. Thus exactly the same infection vectors are used as a typical user would experience in everyday life. The test-cases used cover a wide range of current malicious sites and provide insights into the protection given by the various products (using all their protection features) while surfing the web.

Age of samples is not taken into account here, their only focus was working samples, which because of age, as stated above, most of the exploits were already patched.

Copyright and Disclaimer:

AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AVComparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data.

Sums it up quite nicely.

 

[ravi prakash saini]

I was not considering the sample used by AV-Comparatives. you can check for yourself just disable the real time av and see the result for BB

 

[212eta]

"Lab Results, again? Seriously, guys...

+1

 

[Korea]

See the discussion on eset here
Any Deep Learning Techniques in ESET products?

And Windows Defender should wait for the next MRG result.
https://www.mrg-effitas.com/wp-content/uploads/2017/05/MRG-Effitas-360-Assessment-2017-Q1_wm.pdf
https://www.av-comparatives.org/wp-content/uploads/2017/06/avc_per_201705_en.pdf

 

[Winter Soldier]

Hmm... sometimes it seems people trust the lab tests just when their favorite products got excellent results, otherwise the tests are not reliable LOL.
But it is not a problem, it is part of human nature to get the best of things

 

[509322]

Read the test methodology. All of the products tested are competent. The differences become apparent when you start executing undetected (FUD) malware on the system. In that case, a product such as Avira or Windows Defender is not going to protect as well as Emsisoft or Kaspersky.