Software Review FS Protection PC 17.9 (beta) test - AVTestGuy - 2020/10/03

[Tachikoma]

Content source

https://youtu.be/mbyp8m81h-Q

I think it did pretty well .

 

[Andy Ful]

I have mixed feelings when watching AV testing videos. The author did pretty well to show how the tested AV looks and works. This test is also consistent with several tests made by AV testing Labs. But, taken alone it cannot say much about AV protection.

The main problem for any particular test is as follows:
hundreds of millions samples in-the-wild ---> MAGIC ---> small number of tested samples

By using MAGIC (even unconsciously) one can show anything and prove nothing.

 

[upnorth]

I have mixed feelings when watching AV testing videos. The author did pretty well to show how the tested AV looks and works. This test is also consistent with several tests made by AV testing Labs. But, taken alone it cannot say much about AV protection.

The main problem for any particular test is as follows:
hundreds of millions samples in-the-wild ---> MAGIC ---> small number of tested samples

By using MAGIC (even unconsciously) one can show anything and prove nothing.

Fully understandable and that's also why it's important with so called, disclaimers that hopefully explains the test better etc.

Personal, I normally enjoy video tests for what they are just as you said, looks and work. For example this video showed me personal some very nice upcoming UI ( user interface ) changes I'm looking forward to in the stable version.

 

[Tachikoma]

I agree with you guys that's why I wanted to show also how the av looks and how it installs

 

[MacDefender]

Overall the results are interesting, I agree, mostly a positive performance. Looks like Avira signatures are doing a good job at static detection but still, some of the samples relied on DeepGuard heuristics/behavior blocking.

One thing that bugs me is that the on-access triggered scans are sensitive to file extensions (like something named .jpeg but not .exe), as you saw in your tests. Even if you set the setting to "all extensions", it still seems like F-Secure makes some decisions around what engines / what kind of file it is based off the extension. I see this frequently when downloading samples from Cape Sandbox where they're just named <sha1 hash>. They don't trigger any detection with an explicit All Detections scan, but the moment I rename it to an .exe, boom, it gets caught. (I assume this is because Avira cloud scanning only triggers on specific extensions)

I did have a question though about the samples you had in the video. I wasn't watching too closely, but some of the things that got through looked borderline. Like there was a KMS38 activator, which I have a local sample of. To the best of my analysis abilities, this is not malware. It does temporarily swap out a few DLLs in system32 with cracked copies in order to fake an activation but then it puts back the original. This behavior (plus it being a piracy tool) result in a lot of engines flagging it. There was also something that looked like a classic DOOM game. Were these samples truly verified to be malware? It's very much possible that you have a trojaned version that mimics the legitimate one I have, just wanted to be sure.

 

[Tachikoma]

Overall the results are interesting, I agree, mostly a positive performance. Looks like Avira signatures are doing a good job at static detection but still, some of the samples relied on DeepGuard heuristics/behavior blocking.

One thing that bugs me is that the on-access triggered scans are sensitive to file extensions (like something named .jpeg but not .exe), as you saw in your tests. Even if you set the setting to "all extensions", it still seems like F-Secure makes some decisions around what engines / what kind of file it is based off the extension. I see this frequently when downloading samples from Cape Sandbox where they're just named <sha1 hash>. They don't trigger any detection with an explicit All Detections scan, but the moment I rename it to an .exe, boom, it gets caught. (I assume this is because Avira cloud scanning only triggers on specific extensions)

I did have a question though about the samples you had in the video. I wasn't watching too closely, but some of the things that got through looked borderline. Like there was a KMS38 activator, which I have a local sample of. To the best of my analysis abilities, this is not malware. It does temporarily swap out a few DLLs in system32 with cracked copies in order to fake an activation but then it puts back the original. This behavior (plus it being a piracy tool) result in a lot of engines flagging it. There was also something that looked like a classic DOOM game. Were these samples truly verified to be malware? It's very much possible that you have a trojaned version that mimics the legitimate one I have, just wanted to be sure.

This is a great feedback thanks ! The point of my videos is in the leftovers that are being detected by Malwarebytes , EEK and HitmanPro as some of the samples that are run are indeed fps like you pointed out .

 

[MacDefender]

This is a great feedback thanks ! The point of my videos is in the leftovers that are being detected by Malwarebytes , EEK and HitmanPro as some of the samples that are run are indeed fps like you pointed out .

Thanks! I love the videos -- it's always great to see these products in action, thank you for taking the time to put these together!