Do you really understand AV test results?

[509322]

I would not be so cruel for AV scoring tests. Some AVs are scoring very well (and some poor) on many different tests over a couple of years. So, for those AVs is very improbable, that it was by chance.
But, I agree that from looking at the results of the one concrete test, nothing interesting can be concluded.
Personally, on my computers I use Defender + hardening, so it is crucial for me to learn its strong and weak points. I think that posting the paen of prise on Defender would be stupid.
Anyway, I like the fact that Defender security is improving.

Microsoft uses "new & improved" Windows 10 security mostly as a marketing gimmick for people to upgrade from no longer supported versions. Microsoft is way behind in security in some respects, at par in some respects, and ahead in some respects. The whole thing just seems weirdly incorporated into Windows from the user's perspective - especially the Exploit Guard. Lots of quirkiness.

 

[Prorootect]

Microsoft uses "new & improved" Windows 10 security mostly as a marketing gimmick for people to upgrade from no longer supported versions. Microsoft is way behind in security in some respects, at par in some respects, and ahead in some respects. The whole thing just seems weirdly incorporated into Windows from the user's perspective - especially the Exploit Guard. Lots of quirkiness.

"Microsoft uses "new & improved" Windows 10 security mostly as a marketing gimmick for people"
- not for me, I prefer my free software choice, my free will.

- this posting...I will think as much as you do.

PS.
Lockdown,
Have you noticed much resemblance between your avatar and mine?
Alfa and Omega, your A is my Lion, your padlock - it's my Lamb!
Everything agrees!

 

[509322]

"Microsoft uses "new & improved" Windows 10 security mostly as a marketing gimmick for people"
- not for me, I prefer my free software choice, my free will.

- this posting...I will think as much as you do.

PS.
Lockdown,
Have you noticed much resemblance between your avatar and mine?
Alfa and Omega, your A is my Lion, your padlock - it's my Lamb!
Everything agrees!

As far as non-volume licensing, Windows is not a money maker for Microsoft when it comes to security.

Microsoft makes money with Home and Pro (non-volume) via the Store and elsewhere.

You didn't get Windows Defender for free. You paid for it when you bought the system - if you bought it from a retailer. Also, Microsoft does not give anything away for free - they keep making money from users through various means during the whole time the users own and actively use the systems and the Microsoft programs.

I do not know why people call Windows Defender "free AV" because it is anything but free. Everybody pays for Windows Defender. You just don't go to a website, pay for it, then download an installer, and then install it. You pay for Windows Defender when you buy the PC from the retailer. Its cost is included in the price of Windows which is included in the price of the unit.

 

[boredog]

But who has added some of the best security pros in the world in the past? I can name a few . EP_EXoff , rootkit developer. Russ R creator of the first rootkit seeing software, rootkit revealer ect. The person that revealed sonys rootkit. No matter what you think about MS they have amazing people working for them. and your buddy Bill did say he would have a pc in every home and the only one that has more money than him is besos, amazon.

 

[509322]

But who has added some of the best security pros in the world in the past? I can name a few . EP_EXoff , rootkit developer. Russ R creator of the first rootkit seeing software, rootkit revealer ect. The person that revealed sonys rootkit. No matter what you think about MS they have amazing people working for them. and your buddy Bill did say he would have a pc in every home and the only one that has more money than him is besos, amazon.

Relevance to what was being discussed ?

Sure, Microsoft has had, does, and will continue to have great employees. So what ? It's a huge, vast corporation. It does a lot of things. A lot of good, neutral, and negative things.

Microsoft, if it wanted to, could hold the entire world for technological ransom right at this very moment. That doesn't make it a great company. That would make it a monopolistic thug (which a large number of people successfully argue that it already is) - actually it would make it a criminal enterprise and not long for this Earth.

The point was, Windows Defender is not free because Microsoft doesn't give anything away for free. There is always some angle that Microsoft plays to make money. People and organizations pay Microsoft - one way or another. Most of the people on these security forums want something for nothing - well you're not getting something for nothing out of Microsoft.

The average Joe would be much better served using a Chromebook. The whole privacy thing with Google is blown way out of proportion. The pundits should take a close look at Windows and what Microsoft does. The average Joe is going to have better overall security on Chromebook if they stay away from Android apps.

 

[Prorootect]

Relevance to what was being discussed ?

Sure, Microsoft has had, does, and will continue to have great employees. So what ? It's a huge, vast corporation. It does a lot of things. A lot of good, neutral, and negative things.

Microsoft, if it wanted to, could hold the entire world for technological ransom right at this very moment. That doesn't make it a great company. That would make it a monopolistic thug (which a large number of people successfully argue that it already is) - actually it would make it a criminal enterprise and not long for this Earth.

The point was, Windows Defender is not free because Microsoft doesn't give anything away for free. There is always some angle that Microsoft plays to make money. People and organizations pay - one way or another.

The average Joe would be much better served using a Chromebook.

- Are there rights in America against monopoly abuse?
surely, then, then

 

[509322]

- Are there rights in America against monopoly abuse?
surely, then, then

Individuals, corporations and the U.S. Justice Dept can pursue such cases, but this is not just a U.S. matter, it is a world matter. Microsoft is a global corporation and not just a U.S corp.

Microsoft has been taken to court many times for anti-trust activities by individuals, other corporations and governments in the past. I'm sure it will happen many times in the future.

Let's not hijack the thread.

 

[TrinitronMSDOS]

Damn ! That's some amazingly detailed and useful infos. Thank you so much for taking the time to write this

As for me, I've never truly believed those tests. I have tested most AVs, and based on my personal (and limited) experience with those, the malwarehub results and independent YouTube review videos, i mostly disagree with the results. I am not saying they are wrong, i'm just saying they are useless to me since they do not reflect what i will notice with said AV on my end.

As an example, not long ago Eset was among the last on AV-Test performance results while Bitdefender was almost first. That does not reflect what i have noticed on many different machines. In AV-Comparatives "real world" tests, Microsoft tend to score up to 98% or 99%. I know they have greatly improve, but i seriously doubt that in a real life scenario Windows Defender would do as good as a job as Bitdefender, Kaspersky or Trend Micro. In fact i think the difference would be very noticeable.

I know there are technical explanations for those differences and that those tests are most likely legitimate. Still, i'm not sure of how useful they are. I mean if the results doesn't reflect what it will be for the users, they are useless for uninformed people. And more tech savvy people already have a pretty good idea of how most vendors perform and which product better suit their needs.

Anyway that's just my 2 cents...

 

[Windows_Security]

This is a true story: I met the AV-Comparatives team on the 14th of december in Innsbruck

Last week I was for business in Innsbruck. Happened to have an appointment in the same building (Soho 2 building on Grabenweg 68 in Innsbruck) on Thursday the 14th of December. So I decided to pay AV-comparatives a visit. They had their company Xmas dinner that evening, so Peter Stelzhammer had an hour and was so kind to show me their office and some of their testing setups.

I confronted him with Lockdown's critism of testing with old samples. He said the real world test were only conducted with the samples cought that month. So the average age of the samples is two weeks. Because AV-comparatibves conducts a special test to check whether the malware is real malware and is still active, their freshest samples are just one to two days old.

He said they condensed malware from 4 million samples a month to about 150 to 350 active and malicious samples. He told me that "when you consider the size of our research team and honey pot network with the resources and infrastructure of AV-companies, AV-comparatives is doing a really good job". The AV-companies often have larger research teams as AV-compartives, and the AV-companies share fresh un-dissected samples between them,

When you know something of statistics, it is a a miracle that AV-comparatives manages to find samples which the combined AV-industry has not found yet. We are not testing with old samples, it is simply bloody hard to to outrun the AV-companies who have large labs and share their catch of the day..When AV-comparatives would work with synthetic samples it would be much easier to cook fresh samples and pass AV-protection. But AV-comparatives performs real world protection tests with real world malware.

Peter understands the dilemma/contradiction of people having some knowledge on security.: AV-companies score very high in real world tests, but when an educated insider changes just a few bits of an existing malware, the protection could drop to 80 percent depending on the existence of generic malware fingerprints for that family. But AV-comparatives does real world tests.It is as simple as that.

 

[Andy Ful]

Yes, YouTube testers definitely have their place as they`re a great way to checkout a programs GUI, system impact, compatibility etc and not just detection results.

Of course the quality of the testers vary from the likes of Cruel Sister, Leo, Malware Geek etc to the more "run of the mill"/awful but yeah I think they do a great job.

P.S. Does anyone know how Panda Free keeps getting such good results as I`ve personally never seen it`s BB in action and it can`t all be down to the excellent Web filter, can it ?

Regards Eck

I would not classify MalwareTips AV videos (like those made by @cruelsister and others) as YouTube videos because they are rather a video-thread type, with discussion and many interesting posts.

 

[Handsome Recluse]

I would not classify MalwareTips AV videos (like those made by @cruelsister and others) as YouTube videos because they are rather a video-thread type, with discussion and many interesting posts.

With possibly two different audiences.

 

[TrinitronMSDOS]

This is a true story: I met the AV-Comparatives team on the 14th of december in Innsbruck

Last week I was for business in Innsbruck. Happened to have an appointment in the same building (Soho 2 building on Grabenweg 68 in Innsbruck) on Thursday the 14th of December. So I decided to pay AV-comparatives a visit. They had their company Xmas dinner that evening, so Peter Stelzhammer had an hour and was so kind to show me their office and some of their testing setups.

I confronted him with Lockdown's critism of testing with old samples. He said the real world test were only conducted with the samples cought that month. So the average age of the samples is two weeks. Because AV-comparatibves conducts a special test to check whether the malware is real malware and is still active, their freshest samples are just one to two days old.

He said they condensed malware from 4 million samples a month to about 150 to 350 active and malicious samples. He told me that "when you consider the size of our research team and honey pot network with the resources and infrastructure of AV-companies, AV-comparatives is doing a really good job". The AV-companies often have larger research teams as AV-compartives, and the AV-companies share fresh un-dissected samples between them,

When you know something of statistics, it is a a miracle that AV-comparatives manages to find samples which the combined AV-industry has not found yet. We are not testing with old samples, it is simply bloody hard to to outrun the AV-companies who have large labs and share their catch of the day..When AV-comparatives would work with synthetic samples it would be much easier to cook fresh samples and pass AV-protection. But AV-comparatives performs real world protection tests with real world malware.

Peter understands the dilemma/contradiction of people having some knowledge on security.: AV-companies score very high in real world tests, but when an educated insider changes just a few bits of an existing malware, the protection could drop to 80 percent depending on the existence of generic malware fingerprints for that family. But AV-comparatives does real world tests.It is as simple as that.

Thanks a lot for the detailed information, i now know a little more about AV testings

Yet another example of why this forum is one of a kind an why i love you guys