AVLab.pl Threat landscape and the results of protection based on telemetry data of malware in the wild (March 2023)

Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

Adrian Ścibor

From AVLab.pl
Thread author
Verified
Well-known
Apr 9, 2018
182
Welcome everyone!

Firstly, if you follow us, it will be good news for you that we have joined AMTSO officially. The May edition will now be official with AMTSO standards.

In the meantime, I have a summary for you from the March 2023 test and some highlights.

Tested software for home and small office:
  1. Acronis Cyber Protect Home Office
  2. Avast Free Antivirus
  3. Avira Antivirus Pro
  4. Bitdefender Antivirus Free
  5. F-Secure
  6. Malwarebytes Premium
  7. Microsoft Defender
  8. Webroot Antivirus
  9. Xcitium Internet Security
Solutions for business and government institutions:
  1. Emsisoft Business Security
  2. Malwarebytes Endpoint Protection
  3. Xcitium ZeroThreat Advanced
In May, we are gonna to include: Kaspersky Plus, Quick Heal Total Security, G Data Total Protection and maybe more. Then, there should be new business solutions.

BETA feature: Individual producer information with Webroot as an example. I wanted to ask you for your opinion, what do you think, is this form of product presentation with details necessary? We have prepared this for you to extract in detail the so-called Remediation Time and maybe other interesting data in the future.

Just FYI - we'll update soon a Changelog website (to be more transparent for everyone) to add another interesting improvements on the backend. These changes has been made and tested to be ready to start the May test edition. For example, we have added an external 3rd party scanner, to better classify malware samples and better reject junks, PUA/PUP - in addition to what we have, i.e. Yara rules, Linux tools and black box analysis. And another one changes is added a new malware in the wild URL source. Be patient and wait for an update.
These improvements will be available for everyone, Vendors and Readers, based on data telemetry from test in May 2023 (a CSV file report with SHA256, as you have it now).

Feel free to ask for anything! :)
Regards, Adrian.
 

Attachments

  • security test in march 2023.jpg
    security test in march 2023.jpg
    1.7 MB · Views: 358

Trident

Level 28
Verified
Top Poster
Well-known
Feb 7, 2023
1,758
BETA feature: Individual producer information with Webroot as an example. I wanted to ask you for your opinion, what do you think, is this form of product presentation with details necessary? We have prepared this for you to extract in detail the so-called Remediation Time and maybe other interesting data in the
Yes Adrian, it is a great feature that adds value to the website. Just in the Webroot case I am not 100% sure they were the first to use the cloud as Panda spoke about collective intelligence very early years. Haven’t investigated Webroot that much in-depth so let’s say it is correct.
 

Adrian Ścibor

From AVLab.pl
Thread author
Verified
Well-known
Apr 9, 2018
182
Thanks for your opinion. The description has been taken from their website and they were the first. Unlike to this, what else would you like to see here?

Maybe:

a. Webroot's (Producer) indicators of antivirus? I mean, exactly which files we're monitoring to get information from the Sysmon about detection and prevention?
b. Additional CSV with malware URL used in the test?
c. Other suggestions?
 

Trident

Level 28
Verified
Top Poster
Well-known
Feb 7, 2023
1,758
Thanks for your opinion. The description has been taken from their website and they were the first. Unlike to this, what else would you like to see here?

Maybe:

a. Webroot's (Producer) indicators of antivirus? I mean, exactly which files we're monitoring to get information from the Sysmon about detection and prevention?
b. Additional CSV with malware URL used in the test?
c. Other suggestions?
I think on the producers pages just these facts and tests are enough. Be careful when copying from websites as they frequently like to claim they are the first and one and only in everything, but then if you fact-check, it is not the case.
You don’t wanna end up with vendors sending you link to patents and nasty emails.
Always prioritise the avoidance of drama first.

The rest of the information you are considering to include like the CSV with the URLs fits your total transparency philosophy and is a great addition. Just maybe not in the producers page, but in the test page. I believe the URLs and the inspection points monitored are for all products, right? Not individual for different products.
I believe the information you include and plan to add is already enough work and more than majority of testing organisations. Maybe a little bit more about the malware as well for curious geeks 🤓
(Not a must).
 
Last edited:

Adrian Ścibor

From AVLab.pl
Thread author
Verified
Well-known
Apr 9, 2018
182
I believe the URLs and the inspection points monitored are for all products, right? Not individual for different products.
Testing database (URLs) is the same for every Vendors and malware telemetry/LOLBINs as well, but each of product has own so-called "indicators" - we call it for example specific file-log when malware is processing, quarantine path, logs path/files, additional tracelogs etc. Just technical things.

I believe the information you include and plan to add is already enough work and more than majority of testing organisations. Maybe a little bit more about the malware as well for curious geeks 🤓
(Not a must).
Yeap. What exactly do you mean?
 

Trident

Level 28
Verified
Top Poster
Well-known
Feb 7, 2023
1,758
Testing database (URLs) is the same for every Vendors and malware telemetry/LOLBINs as well, but each of product has own so-called "indicators" - we call it for example specific file-log when malware is processing, quarantine path, logs path/files, additional tracelogs etc. Just technical things.
Alright, so the URLs can go on the test page and the logs can go on the vendor page, I believe this is the most logical in this case.
Yeap. What exactly do you mean?
I mean you already include plenty of details. Maybe the threat family but you’ve said already that some third-party scanners will be integrated.

Btw ZoneAlarm was tested at one point, will we see it again? And Norton maybe as well?
 

Trident

Level 28
Verified
Top Poster
Well-known
Feb 7, 2023
1,758
So you are the standard of what is correct an what is not? ;)
Not sure why you are creating this off-topic post here without adding any value to the thread, across any of the threads and posts I have created, I’ve never made claims I was the standard of what’s correct or the information posted is the “ultimate truth”. This is a community forum where people come to post what they believe they know and not a university assignment or a Wikipedia article.

In the Webroot case I am not convinced they are the first and when publishing facts like this online, on their own website, one must be careful and avoid any potential problems.

If you don’t like my posts, there is an “ignore” button conveniently situated at the bottom of each one of them.
 
F

ForgottenSeer 97327

In the Webroot case I am not convinced they are the first and when publishing facts like this online, on their own website, one must be careful and avoid any potential problems.
PrevX was the first, but Panda included CLOUD in their product name. Would not you trust a professional like Adrian to fact check content before putting it on his website
 
Last edited by a moderator:

Trident

Level 28
Verified
Top Poster
Well-known
Feb 7, 2023
1,758
PrevX was the first, but Panda included CLOUD in their product name. Would not you trust a professional like Adrian to fact check content before putting it on his website?
Panda had collective intelligence long before the Cloud version was released and who knows how many others have had different implementations of that as well. The cloud version replaced signatures with a small cache of malicious hashes but that was only in 2008-2009.
Bitdefender had “outbreak detection” since the very early years as well and spoke about NeuNet (neural networks) as early as 2006-2007.
Statements that Webroot was the first to include cloud and AI are difficult to check and better to be avoided. Specially when OpenText is deemed a patent troll.
 
  • Like
Reactions: roger_m
F

ForgottenSeer 97327

@Trident Again: Would not you trust a professional like Adrian to fact check content before putting it on his website?
 

Adrian Ścibor

From AVLab.pl
Thread author
Verified
Well-known
Apr 9, 2018
182
OK, Gentleman. Let's say the Webroot was one of the first :)

Alright, so the URLs can go on the test page and the logs can go on the vendor page, I believe this is the most logical in this case.
On second thought, this information will be addresses to technical geeks. We need to think whether this will be interesting at all:

HKLM\SOFTWARE\WOW6432Node\WRData\Status\Infected
HKLM\SOFTWARE\WOW6432Node\WRData\Status\ThreatBlocked
HKLM\SOFTWARE\WOW6432Node\WRData\Status\ThreatsRemoved
HKLM\SOFTWARE\WOW6432Node\WRData\Status\WasJustInfected
HKLM\SOFTWARE\WOW6432Node\WRData\Threats\*
HKLM\SOFTWARE\WOW6432Node\WRData\Threats\Active\Count
HKLM\SOFTWARE\WOW6432Node\WRData\Status\CurrentlyCleaning
HKLM\SOFTWARE\WOW6432Node\WRData\FileFlags
C:\ProgramData\WRData\WRLog.txt
C:\ProgramData\WRData\WRLog.log !!! So important for unknown files
C:\ProgramData\WRData\ace1.db or \ace*
C:\ProgramData\WRData\dbk.db or \dbk*

So, these are so-called Webroot's Antivirus Indicators - when some of them is triggered and logged by Sysmon, we know that malware is or was processing successfully.

*edited*
If you want to test it or similar indicators for another AV at own machine, please do not forget to set a Sysmon driver altitude regarding with our methodology (recommended): Methods Of Carrying Out Automatic Tests » AVLab Cybersecurity Foundation
Code:
To change the Sysmon driver altitude we suggest the following CLI command:

reg add "HKLM\SYSTEM\CurrentControlSet\Services\SysmonDrv\Instances\Sysmon Instance" /v Altitude /t REG_SZ /d 244999

I mean you already include plenty of details. Maybe the threat family but you’ve said already that some third-party scanners will be integrated.
OK, let's see how it goes after the May edition.

Btw ZoneAlarm was tested at one point, will we see it again? And Norton maybe as well?
I do not say - NO. maybe in next upcoming months. CheckPoint has changed Kaspersky's engine to another one, therefore it might be interesting to see their product on the list.
 

Trident

Level 28
Verified
Top Poster
Well-known
Feb 7, 2023
1,758
OK, Gentleman. Let's say the Webroot was one of the first :)
Yeah, using fuzzy statements like this is great from legal point of view.

You’ve really spent time monitoring Webroot and others. This information for me personally is extremely interesting, but let’s hear what other people think.

I do not say - NO. maybe in next upcoming months. CheckPoint has changed Kaspersky's engine to another one, therefore it might be interesting to see their product on the list.
They have a lot of engines, what is called “Anti-Malware” seems to to be to Sophos. Apart from that they use static analysis, and the emulation uses their own proprietary signatures + Bitdefender. They will be interesting to see. Norton too.
 

Muddy7

Level 2
Verified
Jun 27, 2014
64
When I moved to Prevx in 2005, it was already saying its whole concept of cyber-protection was increasingly going to be built on a database in the cloud collected from all malware activity detected on and eliminated from its users' devices.
OK, they were not yet using the word "cloud" as this was not yet a thing in cybersecurity and still somewhat nascent in IT in general — they used other wording that I can't remember exactly, something like "community intelligence" or something like that — but frankly this period (circa 2005) was long before Panda or others were talking or even thinking about the cloud for cybersecurity. Yes, Prevx (acquired in late 2010 by Webroot) was indeed the first AV company to use the cloud and, indeed, use its cloud malware database as the foundation of its cyberprotecion.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top