AVLab.pl Product Of The Year 2025 – Summary of Advanced In-The-Wild Malware Test

[Adrian Ścibor]

Dear Community!

I wanted to invite you to read a summary of the year 2024. By the way, this is probably a good explanation of our tests, methodology, how we proceed, how we calculate results, etc.

And, of course, I hope for your feedback, comments, so that we can improve in 2025

So without prolonging, I provide the link to download PDF: Product Of The Year 2025 – Summary Of Advanced In-The-Wild Malware Test » AVLab Cybersecurity Foundation

 

[Bot]

Thank you for sharing this valuable resource! It's great to see the detailed methodology and results from 2024. I'm sure many members of our community will find this information beneficial and provide insightful feedback for improvements in 2025.

 

[badboy]

Dear Community!

I wanted to invite you to read a summary of the year 2024. By the way, this is probably a good explanation of our tests, methodology, how we proceed, how we calculate results, etc.

And, of course, I hope for your feedback, comments, so that we can improve in 2025

So without prolonging, I provide the link to download PDF: Product Of The Year 2025 – Summary Of Advanced In-The-Wild Malware Test » AVLab Cybersecurity Foundation

Тhank you for testing! Polecam!

 

[simmerskool]

Thank you for sharing this valuable resource! It's great to see the detailed methodology and results from 2024. I'm sure many members of our community will find this information beneficial and provide insightful feedback for improvements in 2025.

@Bot can you summarize the report, I skimmed it and it looked like there was more than 1 product of the year...(or I need new glasses)

 

[mlnevese]

@Bot can you summarize the report, I skimmed it and it looked like there was more than 1 product of the year...(or I need new glasses)

I think everyone was product of the year after reading it...

I asked Claude AI to analyze and tell me who the winner was just to check if I had lost the ability to read and here is its report:

Let me analyze the report carefully to determine the Product of the Year 2025 winners.

According to the report's award criteria, to win the "Product of the Year 2025" award, a solution needed to:
1. Participate in at least 3 editions of the Advanced In-The-Wild Malware Test
2. Obtain 3x EXCELLENT certification, demonstrating a minimum of 99% protection effectiveness

Looking through the test results for each product, here are those that met these criteria:

DEFINITE WINNERS:
- Malwarebytes Premium - achieved 100% protection across all 6 test editions
- ThreatDown Endpoint Protection - achieved 100% protection across all 6 test editions

LIKELY WINNERS:
- AVAST Free Antivirus - achieved 100% protection across all 6 test editions
- Emsisoft Enterprise Security - achieved 100% protection across all 6 test editions

However, I should note that while the report provides detailed test results, it doesn't explicitly state the final award winners. The above products met the stated criteria based on their test results, but the actual award winners might vary if there were additional unstated criteria or considerations.

 

[Vitali Ortzi]

@Adrian Ścibor can you try using tdss killer or any trust binary , or even uploading a binary to get trusted by xcitium and then use that trusted binary to load a malicious dlls as that's basically the method they use to bypass comodo, xcitium client and please try to see how you can enhance as much as possible the ATP attacks this year and try to see if you can get checkpoint products into the tests as they have improved over the years and get pretty good results in shadowra tests in this fourm (second only to microsoft ATP in the enterprise test he made ) and generally checkpoint gets either good results or some of the top results in third party lab tests

Note although I said above comodo has a flaw there is no reason someone shouldn't use it to complement his av software as I myself use it alongside av software I have on my machines such as eset , Microsoft etc
As comodo with cruelsister settings can stop almost every malware except ones abusing trusted processes in certain tactics

 

[Adrian Ścibor]

@Adrian Ścibor can you try using tdss killer or any trust binary , or even uploading a binary to get trusted by xcitium and then use that trusted binary to load a malicious dlls as that's basically the method they use to bypass comodo, xcitium client and please try to see how you can enhance as much as possible the ATP attacks this year

The Advanced In-The-Wild Malware Test is not suitable to this specific requirement, but rather EDR-XDR evaluations that will be performed in 2025.

and try to see if you can get checkpoint products into the tests as they have improved over the years and get pretty good results in shadowra tests in this fourm (second only to microsoft ATP in the enterprise test he made ) and generally checkpoint gets either good results or some of the top results in third party lab tests

Note although I said above comodo has a flaw there is no reason someone shouldn't use it to complement his av software as I myself use it alongside av software I have on my machines such as eset , Microsoft etc
As comodo with cruelsister settings can stop almost every malware except ones abusing trusted processes in certain tactics

I am afraid we do not have access to the Check Point Harmony so far. We tested ZoneAlarm in January 2025, the results will be published in two weeks, perhaps we can get access to CPH in March 2025 in the next edition.

 

[BulletKnowledge]

Product of the year? All product because missed sample is an exception and the standard is 100% result.

I asked Claude AI to analyze and tell me who the winner was:

DEFINITE WINNERS:
- Malwarebytes Premium - achieved 100% protection across all 6 test editions
- ThreatDown Endpoint Protection - achieved 100% protection across all 6 test editions

LIKELY WINNERS:
- AVAST Free Antivirus - achieved 100% protection across all 6 test editions
- Emsisoft Enterprise Security - achieved 100% protection across all 6 test editions

Why Claude AI made two category when the result is the same in the two category?
Why Claude AI omit Comodo and Xcitium when they have the same result than the one in this two category?

 

[mlnevese]

Product of the year? All product because missed sample is an exception and the standard is 100% result.


Why Claude AI made two category when the result is the same in the two category?
Why Claude AI omit Comodo and Xcitium when they have the same result than the one in this two category?

Good questions but I'd have to redo the questions to know the answer. And that's even considering it would give the same answer this time

 

[simmerskool]

Good questions but I'd have to redo the questions to know the answer. And that's even considering it would give the same answer this time

ask DeepSeek

 

[mlnevese]

ask DeepSeek

I did... Here is the answer, with a screenshot attached before someone accuses me of inventing answers:

According to the report, the winners of the "Product of the Year 2025" award are not explicitly listed in the provided content. The document outlines the criteria for the award and describes the testing process, but it does not specify which products or companies received the "Product of the Year 2025" or "TOP Remediation Time 2025" awards.

To find the specific winners, you would need to refer to the full report or the section that details the results of the Advanced In-The-Wild Malware Test in 2024, which may be available on the AVLab website or in the complete document.



Also Claude gave me a more detailed answer this time.



And when I asked more details:




Full transcript:

I'll analyze which products were eligible and most likely to win based on the stated criteria.

First, let's identify products that participated in at least 3 editions:

1. Avast Free Antivirus - All 6 editions, 100% protection in all tests
2. Comodo Internet Security - All 6 editions, 100% protection in all tests
3. Emsisoft Enterprise Security - All 6 editions, 100% protection in all tests
4. Malwarebytes Premium - All 6 editions, 100% protection in all tests
5. ThreatDown Endpoint Protection - All 6 editions, 100% protection in all tests
6. Webroot Antivirus - All 6 editions, 99.96% protection (one failure)
7. Xcitium ZeroThreat Advanced - All 6 editions, 100% protection in all tests

Products that participated in 4 editions:
- Bitdefender Total Security - 4 editions, 99.9% protection (2 failures)
- ESET Smart Security Premium - 4 editions, ~99.9% protection (3 failures)
- McAfee Total Protection - 4 editions, 100% protection
- Quick Heal Total Security - 3 editions, 100% protection

Products with only partial participation or significant failures:
- F-Secure Total - All editions but had failures (99.93%)
- Panda Dome Advanced - All editions but had multiple failures (99.59%)

Based on the criteria of:
1. Minimum 3 editions participation ✓
2. 99% minimum protection effectiveness ✓
3. 3x EXCELLENT certification

The strongest candidates appear to be:
- Avast Free Antivirus
- Comodo Internet Security
- Emsisoft Enterprise Security
- Malwarebytes Premium
- ThreatDown Endpoint Protection
- McAfee Total Protection
- Xcitium ZeroThreat Advanced

These all maintained 100% protection across their test periods. However, the report doesn't specify which products received the "EXCELLENT" certification three times, which is the final criterion needed to definitively determine the winner.

Would you like me to analyze any specific aspects of these top performers in more detail?



These AI tools can be very useful sometimes.

 

[Digmor Crusher]

Interesting, I guess AI isn't as useless as I thought.

 

[BulletKnowledge]

Interesting, AI can be good i agree.

 

[mlnevese]

Interesting, AI can be good i agree.

It's often used in my field for document analysis. It can generate excellent reports as long as you don't push it into hallucinations—like asking for its opinion, for instance. However, you can send it a contract and ask it to list all parts that affect privacy, for instance.

It's also great for grammar checking and is often better at translation than translation sites. It can also rewrite something you've written in a formal and educated way, for instance—perfect for when you're too furious about something to write in a coherent, calm manner

 

[Adrian Ścibor]

To find the specific winners, you would need to refer to the full report or the section that details the results of the Advanced In-The-Wild Malware Test in 2024, which may be available on the AVLab website or in the complete document.

AI is useless in a bit more complex matters We proved it in the test, which is in Polish: Czy AI zastąpi specjalistów IT? Test sztucznej inteligencji cz. 2 » AVLab.pl (Will AI replace IT professionals? Artificial intelligence test part 2.)

The solutions that received the certificates contain information in the form of graphics, not text in PDF. You should stop using AI, because it is not suitable for everything. It can't reach conclusions too

 

[simmerskool]

It's often used in my field for document analysis. It can generate excellent reports as long as you don't push it into hallucinations—like asking for its opinion, for instance. However, you can send it a contract and ask it to list all parts that affect privacy, for instance.

It's also great for grammar checking and is often better at translation than translation sites. It can also rewrite something you've written in a formal and educated way, for instance—perfect for when you're too furious about something to write in a coherent, calm manner

fwiw chatgpt (paid) "upgraded" me to o3 (& o3 mini) based on its output today I haven't been able to determine if it is better than o1 -- meanwhile deepseek is not accepting my email, and have no intent to put its app on my iphone.

 

[mlnevese]

AS I said, AIs are a useful tool in document analysis, not the only one. They have gone a long way since the first public releases.