AVLab.pl Analysis of system protection against active online malware – July 2025

Status
Not open for further replies.
Disclaimer
  1. This test shows how an antivirus behaves with certain threats, in a specific environment and under certain conditions.
    We encourage you to compare these results with others and take informed decisions on what security products to use.
    Before buying an antivirus you should consider factors such as price, ease of use, compatibility, and support. Installing a free trial version allows an antivirus to be tested in everyday use before purchase.

It's probably Nairobi or the Maldives.
If i was a dishonest person, I would make my Command and Control endpoints either with major cloud providers, hijacked websites, or dormant domains.
Cool. We are up to page 8 on my browser.
 
  • Like
Reactions: simmerskool
Adrian- First off, compliments for your site. The fact that you provide a detailed spreadsheet of the malware used with product results speak highly of its integrity.

The issue that I found was with the Comodo result. Although Xcitium relies on an extent on Valkyrie, Comodo does not; therefore, any error (whitelisting of the sample) in the Valkyrie Cloud would not apply to Comodo. The only possibility would be if this sample was added into the LOCAL File Rating database which could not have occurred as there has been no program updates for a while.

My posts are about Comodo (normally the Firewall only setup) without the AV module. Further, when running test malware in Comodo I always disable both the Cloud AV (VirusScope) and File rating components, thus relying only on the strength of the product which are the Sandbox and firewall modules.
Thus, when the sample is run it will go into the sandbox, and not being able to establish a connection to a Server will close. Further, even with Containment disabled the Firewall will block the malicious connection out to Moscow which also results in the malware closing.

In Summary, the statement on your website:

"The results of Comodo and Xcitium are also surprising, as both solutions failed to block a single (same) threat that had previously been classified by the developer as a False Negative: incorrectly marked as safe in the developer’s cloud, which may have affected the global response for workstations."

in actuality may ONLY apply to Xcitium and NOT Comodo, the latter not relying on the Valkyrie Cloud at all.

Regards,
Meghan

Apologies in advance, but it's not my job to argue about who is right.

1. The logs were sent to the developer BEFORE the results were published. For example:

tree.png

img: process correlation tree over time
events.png

img: other details based on logs

2. The developer confirmed that the result is negative for this sample for both solutions.

3. The results were published without doubts from the developer.


My goal is not to prove who is right and who is wrong. Maintaining professionalism here, we rely on logs. The logs tell us what happened and what did not happen.

We did not record any Sandbox or other module reactions in the logs that could change the result for the sample. The remaining details are available to the vendor for a fee.

The problem with this sample has been resolved by the developer, so when analyzing the results after the fact, it should be marked as malicious in the Valkyrie cloud.
Ultimately, at AMTSO, we have to rely on logs and consult certain things with developers, which makes AVLab tests more professional than tests on YouTube.
Other issues are discussed systematically, such as results that are always close to 100%, to which we respond that after the new year, we will change the awarding of solutions to a more rigorous system.

If there has been any undesirable behavior in Comodo or Xcitium, it is not our job to prove it, as we are not software developers, but we are happy to work with the developers to clarify certain issues. This stage is not included in amateur tests.

A fail result for the sample has already been resolved before the results were published, so consider it a positive thing that there are people there who are fixing problems, bugs.
 
Last edited:
in actuality may ONLY apply to Xcitium and NOT Comodo, the latter not relying on the Valkyrie Cloud at all.
2. The developer confirmed that the result is negative for this sample for both solutions.
I think Valkyrie verdicts are part of Comodo Cloud. Melih, I believe, confirmed that Comodo Cloud offers Valkyrie verdicts in response to questions regarding the promised Valkyrie feature missing in Comodo 2025. The HIPS log should have an entry about the sample's verdict, if I'm right.
 
  • Like
Reactions: piquiteco
I think Valkyrie verdicts are part of Comodo Cloud. Melih, I believe, confirmed that Comodo Cloud offers Valkyrie verdicts in response to questions regarding the promised Valkyrie feature missing in Comodo 2025. The HIPS log should have an entry about the sample's verdict, if I'm right.
file rating.png


It may use the Valkyrie cloud for File Rating, but I am not 100% sure. Link to the privacy policy for sharing file metadata: Privacy Policy | Policies and Practices of Comodo Cybersecurity
 
  • Like
Reactions: New_Style_xd
Not at all unusual for social media. Forums are social media. Forums are social media with topic threads.

Perhaps MT can adopt the Reddit format of a running, "no page" thread style.
In the past, forums of all kinds were very popular. People sought specialist help. They wrote about their hobbies.

Then everything spilled over into social media. Thematic and technical discussions were lost because it is not possible to quote, insert code, or write in-depth analyses.

Forums are a very good thing. MT should remain in this form because people need it. They will return to it from social media, which is full of ads, where nothing is yours, your content is not your property, and can be deleted, banned, or censored at any time.
 
View attachment 290637

It may use the Valkyrie cloud for File Rating, but I am not 100% sure. Link to the privacy policy for sharing file metadata: Privacy Policy | Policies and Practices of Comodo Cybersecurity
As far as I recall... CIS gets Valkyrie verdicts via Cloud Lookup. Comodo uses Valkyrie to analyze submitted files. Xcitium has near real-time Valkyrie analysis. Comodo receives a Valkyrie verdict after human analysis. The business version of Valkyrie has extra features.
 
  • Like
Reactions: piquiteco
Wow, amazing the affect one 1 false negative / missed sample can do.

Glad the tests are still ongoing and gives a view of how effective products are.
 
Last edited:
I'm waiting to see what cruelsister's new excuse will be...
She doesn't need an excuse; Comodo continues to be an effective solution for the majority. The issue here, a false negative, is not uncommon; mistakes happen.

And the worst part is that there are still people who believe they are safe.
You and a few other users are constantly active on the Comodo forum, posting and reiterating the same points repeatedly. The worst part is knowing that a product has ongoing issues and that the vendor has neglected to resolve these problems for years. It seems unlikely that they will address them now or in the future, yet you still choose to hold on to the product.
 
Dude, I think that now there are no arguments against facts.
And the worst part is that there are still people who believe they are safe.
Obviously Comodo has some objectionable aspects, both as a company and a product, but obsessing over exaggerated views of complete inadequacy doesn't make sense. I still think it's important to inform people that it has shortcomings that won't be fixed. A home user could nevertheless enjoy satisfactory or above-average protection with Comodo properly set up if they're so inclined. I use a security product engineered and continuously maintained by a multibillion-dollar company and could still be vulnerable to malicious code.
 
She doesn't need an excuse; Comodo continues to be an effective solution for the majority. The issue here, a false negative, is not uncommon; mistakes happen.
The fact is there is a cadre of people here that just don't like Cruelsister and want to shut her up or, better yet, get her to close her MT account permanently.

@cruelsister, your style of not engaging in debates to fully defend your position(s) works against you here (and online in general). Posting a video and then going away and not addressing your detractors one-by-one in each and every instance gives the appearance that you are "blindly promoting Comodo to the detriment of others." Short, one-off responses are perceived by little minds as no response at all and generate the expected responses of "See, she does not take this seriously and, as a fanatic, is deliberately avoiding the criticism(s) made."

The online world has changed. Regardless of whether your critics are smart and articulate or their minds are dreck, it is perception that prevails and matters.

If you have a position, then you will have to defend it actively, in detail, and vigorously. If not, nobody will care to draw conclusions from a video post and then a thread with no consistent active defense of it on your part.

Things should not be this way. It's extremely unfortunate, but people are people. What can I tell you other than "The Hoomans are a scourge and the Romulans were correct about them."

I know this is not your style and you're disinclined to do it, but there it is.

If you are as I am - "It's mind over matter, I don't mind because y'all don't matter," then - I salute you - fellow Gladiator. Haters gonna hate and they can rot.
 
@cruelsister, your style of not engaging in debates to fully defend your position(s) works against you here (and online in general). Posting a video and then going away and not addressing your detractors one-by-one in each and every instance gives the appearance that you are "blindly promoting Comodo to the detriment of others." Short, one-off responses are perceived by little minds as no response at all and generate the expected responses of "See, she does not take this seriously and, as a fanatic, is deliberately avoiding the criticism(s) made."
Or maybe she's just too smart to get engaged in that sort of stuff that goes no where and usually results in a thread being closed.
 
Or maybe she's just too smart to get engaged in that sort of stuff that goes no where and usually results in a thread being closed.
It could be that and/or she doesn't care. Or both. I was merely explaining the perception. She has no obligation to do anything and I'm certainly not advocating that she respond per what I described. This forum is social media not matter what anybody says and the behaviors of "members" that don't like Comodo or @cruelsister are predictable.

Some people just cannot help themselves from bashing Comodo at any opportunity and those that support it (the "immoral fanatics").

No software publisher owes anything anything, it doesn't have to do anything, unless there is a contract. It has no social or ethical acceptability obligations. It doesn't have to fix bugs. It does not have to fix security issues. It doesn't have to innovate. It can abandon its software. And fanbois/fangirlz can promote it all that they so choose even if, technically, its abandonware with multiple flaws.

There are others that refute or disagree with most of that, and it is their right and entitlement to do so. They just cannot be taken seriously no matter how well meaning their intent or how butthurt they are about the publisher, the promoter(s), and the dedicated users.

The marketplace must remain the final arbiter of what is good, and more importantly what is not. That duty should never fall to critics and anti-product activists.

MT is social media. Plain and simple. We get what we get here.
 
  • Like
Reactions: Trident and roger_m
Yikes! Finally have time to APOLOGIZE for making a GRAVE MISTAKE! Although I ran the sample with the nest of intentions I initially ran it under different conditions than did Adrian.

Whenever I test a new sample, I prefer just to run the file into the strength of Comodo (Sandbox and Firewall); to this end I routinely first turn off both VirusScope and File Rating. Under this condition the malware is indeed contained and prevented from running.
Secondly I will enable both VirusScope and File Rating and run the sample again. Currently Comodo cloud will detect and delete the malware (Thanks ONLY to Adrian's contacting them). So in BOTH cases resulted in a verdict that differed with the reality that existed when AVLab conducted their most excellent test.

Sadly I tested the sample after Comodo made the change of removing a mistaken WhitleListing, and under the condition of having both File Rating active. I should have seen this sooner as I've been anything but a Fan of Dumb (typical AV database) detections.

So to sum up. I apologize to the MT Board. Adrian, and AVLab.pl.
 
I apologize to the MT Board
In my estimation, the "MT Board" does not include MT staff.

Tolerance by "members" of the "MT Board" is non-existent. Plus there's a high degree of ignorance. And hence, you have to forgive them because they can't help themselves.
 
  • Like
Reactions: Trident
And the worst part is that there are still people who believe they are safe.
If they believe they are safe, then isn't that wonderful for them?

Why do you care so much what others believe or do not believe? It's their data, their money, and I could care less what happens to any of it. Ethics? Social responsibility? That's complete BS.

Even if you presented those that believe irrefutable evidence, they're extremely likely to disregard that evidence.

Besides, it is everybody's prerogative to live as dangerously as they want, whether or not they know they're living dangerously. Just go to any fast food location and you'll observe people stuffing their faces, knowingly killing themselves - slowly but surely. Same principle applies to any security - physical, digital, or otherwise.
 
  • Like
Reactions: Trident and roger_m
Status
Not open for further replies.