Battle Which shoud I trust, AV-Comparatives or AV-TEST? And should I use Browsing Protection?

[kamo_jisan]

Hello all members! Thank you for your access here.
Just a moment, would you use your time a little for solving my question?

There are some famous test for resarch of ability to protect devices by anti-malware softwares.
For example, AV-TEST and AV-Comparatives are the most famous test of the world.

Last week, I was looking the result of them, which tested anti-malware solutions to protect from malwares.
I have accessed to the result; AV-Comparatives's Protection Test and Real-World Protection Test, and AV-TEST results.

But I had a question about these showing.
According to Real-World Protection Test by AV-Comparatives, Bitdefender is better than Avast, and even Kaspersky. Even Microsoft is also better than them.
But according to Malware Protection Test by AV-Comparatives, Avast is better than Bitdefender. And the MS is not good.
And also that AV-TEST is showing the test result in February 2017. At the result, Avast Free, Bitdefender Internet Security, and F-Secure SAFE are rated as full point at Protection section.

So I want to ask everyone that which results shoud I trust?
And...

<Question No.1>
I am having some license; Bitdefender Total and F-Secure SAFE. And I am also ble to install Avast Free or Fortinet, SOPHOS, Avira Free.
Which software is it excellent for protect my PC from threat (From using Browsers, Email, and more)?
What do you think?

<Question No.2>
I think that Avast Free is very nice soft for many consumers like me because of light and high score protection.
But Bitdefender is rated as one of the best solution for keeping security of PC.
As saying before, which software is better for keeping security?

<Question No.3>
According to test result, Bitdefender Internet Security and Total Security are one of the best.
By the way, as saying before, I am having F-Secure SAFE license as trial of it.
If you choose security solution from these two softwares; Bitdefender Total or F-Secure SAFE, which will you choose?

<Question No.4>
Both software, Bitdefender and F-Secure are using Bitdefender Engine, and both are given high score at Protection ability.
I think that F-Secure is good for me because of light and high rate protection.
But it blocks many website as dangerous site based in false positives.
It is too inconvenience for me, so if I use F-Secure SAFE, I turn off Browsing Protection feature(Dangerous site are blocked by this feature. I will use Browsing Scan Feature without blocking site feature).
If I use F-secure SAFE without the feature of "Browsing Protection", Bitdefender Total (or Internet Security) is better than F-Secure SAFE for me?
Is the F-Secure SAFE's Browsing Protection feature indispensable for keeping PC security? I can not decide it.

My PC: Windows 10 Home/ Core i7/ 8GB RAM/ HDD Storage

By the way, I am not able to write English enogh correctly, so please tell me it if you can not understand my sentence.
I hope to your participation!
Thank you very much.

 

[Deleted member 178]

But I have serious queries on what they call as bypass.

And i have serious doubts about their methodology...

 

[Orion]

And i have serious doubts about their methodology...

You are not alone.I had posted a link to the malware protection test topic which you can go back and read.I tried bringing the points close to real world as much as possible.Let me know your thoughts.

 

[Deleted member 178]

Quoting your post over there:

1)Are these samples the same ones used in real world test where you pasted the download link in the browser? If yes then why use a sample whose URL is already blocked.

Would be pointless indeed...

2)Has AV-C filtered out the samples because there is a very less amount that has the ability to spread by USB.You guys should be using malicious emails now because they are the biggest way of spreading ransom malware.

I think they just focus on "on-access" detection rather than "on-execution."

3)There is no clear statement of how they consider the sample as bypassed? Eg: If AV blocks and detects the dropped/downloaded binary the system is still protected even if the dropper runs or downloads this is usually applicable for ransomware which spreads from JS downloaders coming from a E-mail client.

Yes and i guess they focus on the droppers only.

4) Don't merge the results with real world graphs even if you plan to do so... the overlap is quite significant and it's more confusing.Making different graphs will allow us to see which AV is good at both or is inevitably getting good results at real world because of it's aggressive URL blocking but doesn't detect too much PE.

Agree

5)Were the samples downloaded on the system in presence of a disabled AV or what? How was it brought onto the system?

I raised the questions on win8/10 because of smartscreen , most of them would be blocked by it , so i guess they disable all native protection, not even sure they use SUA.

6) If you want to perform this type of a test which is great in my view please do it with threats that come via email and USB specifically .Don't just ship a ton of binaries onto a stick or a system without knowing whether the samples really do spread via a USB or not.

Agree, but again what are their priority ? detection or execution?

 

[Orion]

"The Malware Protection Test is an enhancement of the File Detection Test which we performed in previous years. It assesses a security program’s ability to protect a system against infection by malicious files; what is unique about this test is that in addition to checking detection in scans, it additionally assesses each program’s last line of defence. Any samples that have not been detected e.g. on-access are executed on the test system, with Internet/cloud access available, to allow features such as behavioural protection to come into play."

On access seems to be more signature oriented which is the wrong way if you have test the entire product.

 

[Deleted member 178]

So the malware protection test is far more informative than the others.

 

[Orion]

So the malware protection test is far more informative than the others.

Or so it would seem.Still there is little of no information of how the samples were bought onto the system and under which conditions.No information on the relation of the samples as well.Detected dropped binary is still a hit in my book.

there is little of no info on how the testbeds are created. All these 99.1% and such scores are not face value. The overlap of the product's detections is not as great as clementi/marx tests suggest.

 

[Deleted member 178]

And they don't even mention the malware; so their test can't be verified. As i said many times, they play the game in favor of vendors because 98.1% is still better than 0% by using real FUD malwares...

 

[Orion]

And they don't even mention the malware; so their test can't be verified. As i said many times, they play the game in favor of vendors because 98.1% is still better than 0% by using real FUD malwares...

I think it's really tough for them to get brand new ones unless they are looking at live feeds from online sandboxes.

And what if the AV companies already got these samples and wrote signatures and AV-c added them later.We are back in testbed construction issues.

 

[kamo_jisan]

Good morning All! And I'm very sorry for my reply, which is too delayed.
I had not be able to post reply to this theread for a few weeks.

By the way, I got many opinions at here.
Some people are saying that Antimalware solution tests are not fare for all solutions due to the business,
and other side people are saying that this awareness is not correct and some tests(e.g. AV-Comparatives...) are trusty.
I can not leach a conclusion that which awareness is correct,
but I got a conclusion that I should not refer antimalware solution tests too far, and some of these tests part are trusty.

I will make all answers to be useful for me to make secure PC. Thanks you!

 

[mekelek]

Good morning All! And I'm very sorry for my reply, which is too delayed.
I had not be able to post reply to this theread for a few weeks.

By the way, I got many opinions at here.
Some people are saying that Antimalware solution tests are not fare for all solutions due to the business,
and other side people are saying that this awareness is not correct and some tests(e.g. AV-Comparatives...) are trusty.
I can not leach a conclusion that which awareness is correct,
but I got a conclusion that I should not refer antimalware solution tests too far, and some of these tests part are trusty.

I will make all answers to be useful for me to make secure PC. Thanks you!

the best reviews you can get are the ones coming straight from the users.
check the threads here, check the videos the users here made testing the products, check the malware samples section, etc.

 

[ravi prakash saini]

@kamo_jisan to confuse you further l have subscription for Internet security from Kaspersky,emsisoft,avast and bitdefender ,however persently I am using comodo internet security and sometimes I do not use any antivirus from any company and still no infection
the first line of defence consists of smart screen+UAC+backup+control on mouse
for second line of defence you can choose any antivirus.
but remember antivirus can save you in one condition only and that is if you can keep your computer free of infections without using any security software
more confuse,you are free to curse me

 

[Game Of Thrones]

the best reviews you can get are the ones coming straight from the users.
check the threads here, check the videos the users here made testing the products, check the malware samples section, etc.

Wrong. Sometimes forums are toward a specific product. From samples to some users, so i would suggest a trusty lab results not a user result

 

[Fritz]

Well said @ravi prakash saini !

Also, adding to the unknown monetary influences, these test always just show a certain reaction to malware at a certain point in time. Add a week and it's a different game.

Throw a coin, getting 5 times heads doesn't mean the next 5 throws will yield 5 times tails. It's still 50/50 each throw.

That said, test are a nice indication where a solution is generally headed. After that, I check forums and test them myself. Some may be buggy for me, like Bitdefender, some may not show the reported behavior (e.g. high false positives with F-Secure, never happened to me).

So read the test results, add a nice whopping of Malwartips and put your personal cherry on top. Enjoy.

 

[S3cur1ty 3nthu5145t]

Wrong. Sometimes forums are toward a specific product. From samples to some users, so i would suggest a trusty lab results not a user result

The problem with trusting these so called professional testing facilities, is not only that they can be swayed by the top security companies quite easily, but their one size fits all methodology is not practical for testing many products that behave and react differently to threats. Not to mention that as users, we are not allowed open access to sample sets to even see if they are legit, in the wild, or even complete and non corrupted. There is no transparency, and you either have to blindly trust, or start testing yourself.

 

[AtlBo]

To put it simply do not trust either of them Looks through the forums here would be a better idea to make an opinion on a AV.

I agree but I think there is an important purpose for independent (uhum...commercial) testing. In theory it can help users know how a program will perform at default settings. It also, and most importantly, puts a-v companies on an edge to make their defaults effective. This is good for PC owners in the general sense. It is a good idea to keep up with the latest PC security news too.

Just my opinion but these independent testing labs are not really aseptic but contaminated by marketing.
From our Hub we get real results.

(see above)

but remember antivirus can save you in one condition only and that is if you can keep your computer free of infections without using any security software
more confuse,you are free to curse me

OK, but the security software industry imo has a valuable role in keeping Microsoft and others with access to a PC honest. Who takes our side on issues about telemetry and so on and where would we be without this voice speaking for us?

The problem with trusting these so called professional testing facilities, is not only that they can be swayed by the top security companies quite easily, but their one size fits all methodology is not practical for testing many products that behave and react differently to threats. Not to mention that as users, we are not allowed open access to sample sets to even see if they are legit, in the wild, or even complete and non corrupted. There is no transparency, and you either have to blindly trust, or start testing yourself.

All of this is legit imo. Maybe it would help if testing organizations could be easily described and their methodologies known easily. I mean, they have credibility so maybe they should use that as Virus Total has done to better define who they are as a testing house (not that VT is a testing house, only that it has achieved credibility that the a-v testers perhaps have not yet achieved albeit in a different realm of the security grid). The methods regularly change which is admittedly made more complicated by the changing threat landscape. It is hard to know with great assurance what a test measures without a great deal of research, something most don't have time to do.

The scientific method always yields valid results, even if the testing methodologies of two tests are different. Yet, if the methods are different, the results mean something different in each case. It would help to know easily and without an hour of research in each case what a test is revealing, especially for the large and most trusted testers. Maybe they should work together to develop a way to create a common and simple presentation method for their testing methods, like a catchy single descriptive term for the test.

 

[Game Of Thrones]

The problem with trusting these so called professional testing facilities, is not only that they can be swayed by the top security companies quite easily, but their one size fits all methodology is not practical for testing many products that behave and react differently to threats. Not to mention that as users, we are not allowed open access to sample sets to even see if they are legit, in the wild, or even complete and non corrupted. There is no transparency, and you either have to blindly trust, or start testing yourself.

Well about sharing the samples as far as i know about them(specially av comparative) they do share them with vendors, i don't think it's logical to share them for public! It's dangerous. Nearly all the names in av industry are participating in these tests and if they don't wanted to they can opt out. I did a research on the prizes that av comparative got, they are legit.

 

[mekelek]

Well about sharing the samples as far as i know about them(specially av comparative) they do share them with vendors, i don't think it's logical to share them for public! It's dangerous. Nearly all the names in av industry are participating in these tests and if they don't wanted to they can opt out. I did a research on the prizes that av comparative got, they are legit.

you can't be biased in the malware samples section, that's what i would call the best place for "reviews".

 

[low L!fe]

I Recommend Bitdefender or Norton