Serious Discussion Why does the Comodo "Disappearing HIPS rules" bug require a complete source code rewrite?

[Trident]

He instructed his few developers at the time to study the various internet security suites available on the market at the time, and to copy the features - much of which was based upon Kaspersky's feature set.

So how did that study go? Cuz I don’t see any resemblance???

 

[bazang]

So how did that study go? Cuz I don’t see any resemblance???

Obviously Melih and his crew went a different way, but they did study Kaspersky Application Control, HIPS, firewall, and so on.

It's a shame that much of the old Comodo forum was made inaccessible when they moved to the new forum. All the answers to the question "Why is Comodo the way it is?" were in that forum. Even if it was accessible, it does not matter. Nobody is going to go back and search for those answers.

One day, CIS/CFW will be no more. And yet, still, there will be outrage and controversy.

Comodo is the security software equivalent of a pixelated nuclear bomb.

 

[Andy Ful]

@Trident,

I think that boot drivers are loaded before many other drivers. So, the "HIPS restore" driver should be a boot driver type evaluated by the ELAM driver.
I do not know the code of the Comodo ELAM driver. It is possible that it does not need information about the boot drivers, but simply evaluates boot drivers against the driver blacklist.
This kind of protection is generally related to ELAM.

 

[Trident]

@Trident,

I think that boot drivers are loaded before many other drivers. So, the "HIPS restore" driver should be a boot driver type evaluated by the ELAM driver.
I do not know the code of the Comodo ELAM driver. It is possible that it does not need information about the "HIPS restore" driver, but simply evaluates boot drivers against the driver blacklist.
This kind of protection is generally related to ELAM.

There was this MsConfig option that saved a boot log with all drivers loading sequence(as well as in safe mode the drivers were screen printed up to Windows 7). But nowadays I am not sure if this option still exists frankly. It used to save the log on C:/bootlog.log or something of this sort.

There are few startup layers in the registry saved as integers.
The ELAM from the antivirus is registered as the first most critical group (value 0) and errors in ELAM (as well as if another driver is called and it’s not stabled) leads to the dreaded inaccessible boot device error (because the kernel attempts to complete the boot process but it doesn’t have file system initialised).

The situation with Elam is very tricky. That power to be the 3rd/4th or whatever (very first in line) comes with responsibility.

The Comodo driver amongst others must contain one function that takes the info from kernel, processes the info and returns the verdict. There is no other way.

 

[Pico]

@GenV did say that in XCS HIPS rules would "pile up" instead of being deleted.
Does this "pile up" mean (after the shutdown bug occurred) that after Windows restart duplicate HIPS rules exist in the HIPS rules list? Duplicate rules seems to me no issue as they can be purged.
I'm just wondering how XCS would tackle it without ELAM driver.

 

[Trident]

@GenV did say that in XCS HIPS rules would "pile up" instead of being deleted.
Does this "pile up" mean (after the shutdown bug occurred) that after Windows restart duplicate HIPS rules exist in the HIPS rules list? Duplicate rules seems to me no issue as they can be purged.
I'm just wondering how XCS would tackle it without ELAM driver.

The ELAM driver has no participation in the tackling.
All drivers can be registered as type 0 and then they will start very early but when exactly, only Microsoft can answer. Malware can register drivers with 0 as well, but they have to be verified by secure boot. If they are simply vulnerable, they will need calls from userland to be abused. Just loading them is not malicious as long as they pass the integrity checks.

They can perform some restorations in userland as well.

I am not sure what “pile up” means but yes. Sounds like duplication (for which they could implement deduplication logics).

 

[Divergent]

There was this MsConfig option that saved a boot log with all drivers loading sequence(as well as in safe mode the drivers were screen printed up to Windows 7). But nowadays I am not sure if this option still exists frankly. It used to save the log on C:/bootlog.log or something of this sort.

There are few startup layers in the registry saved as integers.
The ELAM from the antivirus is registered as the first most critical group (value 0) and errors in ELAM (as well as if another driver is called and it’s not stabled) leads to the dreaded inaccessible boot device error (because the kernel attempts to complete the boot process but it doesn’t have file system initialised).

The situation with Elam is very tricky. That power to be the 3rd/4th or whatever (very first in line) comes with responsibility.

The Comodo driver amongst others must contain one function that takes the info from kernel, processes the info and returns the verdict. There is no other way.

You've perfectly described the classic in-kernel security model, and you're right, in that specific architecture, there really is no other way for it to work. The kernel needs to explicitly call a trusted driver (like ELAM) that is running at the same privilege level to get a verdict on other components.

However, modern security has introduced a fundamentally different approach that processes this information in another way, "hypervisor-based security".

The "Referee Outside the Ring" Model

Instead of having the security driver operate "inside" the kernel (Ring 0), this model places a hypervisor (a thin layer of software) directly on the hardware, underneath the main operating system. This creates a more privileged layer, sometimes called "Ring -1," from which to oversee the OS.

Here’s how the information processing is completely different:

Isolation

The security and validation logic is entirely isolated from the Windows kernel. The hypervisor controls the memory and hardware resources allocated to the main OS, acting as an impartial and untouchable referee.

Interception, Not Callback

In the ELAM model, the kernel makes a "callback" to a function inside the Comodo driver. In the hypervisor model, there is no callback. Instead, the hypervisor "intercepts" the kernel's attempt to load any driver. It essentially pauses the OS, inspects the driver from the outside, and then decides whether to let the operation proceed.

External Verdict

The hypervisor makes the allow/block decision independently. If a driver is malicious or untrusted, the hypervisor simply refuses to let the OS execute it. The kernel doesn't get a "verdict", the malicious operation is just stopped before it can even fully execute.

This is the principle behind technologies like "Hypervisor-Protected Code Integrity (HVCI)" in modern Windows. It's a more robust security model because even if the entire OS kernel were compromised by malware, the malware would have no way to tamper with the hypervisor protecting it from underneath. It directly addresses the "power comes with responsibility" issue you mentioned, by moving that ultimate power outside of the ring altogether.

In short, the hypervisor wouldn't allow any unauthorized process, including a malfunctioning part of Comodo's own software, to alter the protected HIPS rules. This architectural change moves the problem from "How do we fix the bug causing the deletion?" to "How do we make the rules impossible to delete in the first place?"

This is precisely why modern security platforms, including Windows itself, are increasingly relying on virtualization to protect their most critical components from both bugs and malicious attacks.

It's a difficult, expensive, and time-consuming path, but it is the correct engineering solution for this class of problem and aligns with the direction the entire cybersecurity industry is moving.

 

[Trident]

But these implementations are still not the standard.

Specially not for Comodo ELAM driver which was probably written years ago and probably never updated again.

I don’t even know if Comodo uses ELAM drivers.

They can’t even write to the registry properly and you want them to implement hypervisors.

Did you not hear Melih that there is no cash for such projects?

Furthermore, this protects against malicious processes. It doesn’t protect if a Comodo-signed process experiences racing conditions on shutdown.
What’s malicious here is a totally flawed, possibly even nonsense logic, potentially coupled with bad design (according to the owner posts) that didn’t follow the basic principles of not writing messy, long, multi-purpose functions, nor did it ensure proper handover from one module to another.

 

[Andy Ful]

The order of driver initialization is usually determined by the Service Group and the dependencies set in the registry.

ServiceGroupOrder:
HKEY_LOCAL_MACHINECurrentControlSetControl\ServiceGroupOrder!List

The group FSFilter Security Enhancer looks interesting because drivers from that group are loaded before the FSFilter Anti-Virus.

Common service groups on Windows 11 (in load order):
System Reserved
EMS
WdfLoadGroup
Boot Bus Extender
System Bus Extender
SCSI miniport
Port
Primary Disk
SCSI Class
SCSI CDROM Class
FSFilter Infrastructure
FSFilter System
FSFilter Bottom
FSFilter Copy Protection
FSFilter Security Enhancer
FSFilter Open File
FSFilter Physical Quota Management
FSFilter Virtualization
FSFilter Encryption
FSFilter Compression
FSFilter Imaging
FSFilter HSM
FSFilter Cluster File System
FSFilter System Recovery
FSFilter Quota Management
FSFilter Content Screener
FSFilter Continuous Backup
FSFilter Replication
FSFilter Anti-Virus
FSFilter Undelete
FSFilter Activity Monitor
FSFilter Top
Filter
Boot File System
Base
(...)

 

[Divergent]

But these implementations are still not the standard.

Specially not for Comodo ELAM driver which was probably written years ago and probably never updated again.

I don’t even know if Comodo uses ELAM drivers.

They can’t even write to the registry properly and you want them to implement hypervisors.

Did you not hear Melih that there is no cash for such projects?

Furthermore, this protects against malicious processes. It doesn’t protect if a Comodo-signed process experiences racing conditions on shutdown.
What’s malicious here is a totally flawed, possibly even nonsense logic, potentially coupled with bad design (according to the owner posts) that didn’t follow the basic principles of not writing messy, long, multi-purpose functions, nor did it ensure proper handover from one module to another.

You're right to point out the practical challenges and the nature of the bug. My suggestion of a hypervisor-based model isn't to downplay the existing issues but to highlight a path to a more robust long-term solution.

You mentioned, "Furthermore, this protects against malicious processes. It doesn’t protect if a Comodo-signed process experiences racing conditions on shutdown". That's a key point.

However, a hypervisor's role isn't limited to just blocking malicious processes. By isolating and protecting critical system resources (like the HIPS rules), a hypervisor can enforce integrity and prevent unauthorized modifications, regardless of whether the cause is a malicious actor or a bug like a race condition. It changes the security paradigm from detection to prevention.

While I understand the resource constraints you've mentioned, continuing to patch a flawed architecture will likely lead to a cycle of recurring bugs. A complete rewrite, while a significant undertaking, is an opportunity to modernize the architecture and build a more reliable and secure product for the future. It’s about moving from constantly fixing symptoms to addressing the root cause.

As for whether Melih will proactively pursue this better method, I think we can both agree that's highly unlikely.

 

[Trident]

As for whether Melih will proactively pursue this better method, I think we can both agree that's highly unlikely.

He told everyone as early as 2011 that he will not pursue such methods (when it was times cheaper) and if a full rewrite was performed, in this day and age legacy, user dependent modules like HIPS likely wouldn’t make it back. Not when Xcitium has the cloud verdict and the containment.

The popularity and revenues of Xcitium wouldn’t be enough for Xcitium to secure a loan given that it’s rather new legal entity as well. Relying just on revenue, they can’t invest more than 5-6% a year from already unimpressive numbers.

It should never have been designed in a way that one module would require a complete rewrite.
However other modules likely require rewriting for other reasons, Melih understands that and this is why he says a complete rewrite will be necessary.

Or it could be just one messy overall platform that is not maintainable and scalable at all.

 

[Andy Ful]

How to fix the HIPS issue without rewriting much of the code.

1. The function that rewrites the HIPS rule should be extended by including what the analogous Xcitium function does. So, CIS will still rewrite the HIPS settings as usual, but additionally, all those settings will be stored one by one in different registry values without rewriting (as a backup).
2. CIS will use an additional kernel driver that compares the HIPS settings with the backup and updates the HIPS settings if they are corrupted.
3. The ELAM driver must be slightly modified to run this additional kernel driver as early as possible before activating HIPS rules.

Some corrections. This is not a full fix, but rather a good mitigation. HIPS can still be corrupted on shutdown or system crash, but it is repaired on Windows startup.
The discussion with @Trident convinced me that the ELAM driver can be unimportant (not sure if possible), because early initialization of the "HIPS restore" driver can be forced by native Windows features.

The mitigation: Use a boot driver to backup/restore HIPS settings.

I think that this mitigation was overlooked by Comodo, or it is unsatisfactory for some unknown reason.

 

[Trident]

Some corrections. This is not a full fix, but rather a good mitigation. HIPS can still be corrupted on shutdown or system crash, but it is repaired on Windows startup.

Software is jam packed with workarounds like this. As long as it works…
This will work.

Potentially even one of the other drivers loading (for example AV or something) can do the same. It doesn’t have to be a dedicated driver. It’s just gonna be something after ELAM.

Problem will be if writing to backup exhibits the same failure (also backup is purged) and you end up with 2 perged configurations.

For this reason it is better to either periodically export all rules to a file as well or diagnose what is causing the error.

 

[Andy Ful]

Problem will be if writing to backup exhibits the same failure (also backup is purged) and you end up with 2 perged configurations.

For this reason it is better to either periodically export all rules to a file as well or diagnose what is causing the error.

If backup is truly incremental, then all backups of HIPS settings before shutdown or crash will survive. The backup must be done in a way that it is flagged as non-corrupted after "creation and an integrity check". When restoring, the backups with no flag are skipped.
I do not think that exporting the rules to a file makes any difference. We already know that HIPS corruption does not affect other Comodo settings, so the backup can be done periodically or by a process independent of HIPS rewriting.

 

[Trident]

If backup is truly incremental, then all backups of HIPS settings before shutdown or crash will survive. The backup must be done in a way that it is flagged as non-corrupted after "creation and an integrity check". When restoring, the backups with no flag are skipped.
I do not think that exporting the rules to a file makes any difference. We already know that HIPS corruption does not affect other Comodo settings, so the backup can be done periodically or by a process independent of HIPS rewriting.

Could be that an erroneous write (some improperly written rule) makes it necessary to purge the configuration and start over… who knows. Could even be something in the data itself that the parsers don’t support.
We can’t parse that, let’s delete all

There are many possibilities.

 

[Andy Ful]

Could be that an erroneous write (some improperly written rule) makes it necessary to purge the configuration and start over…

Yes. The possibly corrupted backups can be removed on Windows startup.

 

[Trident]

Yes. The possibly corrupted backups can be removed on Windows startup.

Yeah but then you have no rules

The purpose of the fix is to force rule retention.

 

[Andy Ful]

Yeah but then you have no rules

What do you mean ?????
You have backups from many days, including the current day. They are uncorrupted in the same way as other Comodo settings.
Only backups from the short period of shutdown may sometimes be corrupted.

 

[Trident]

What do you mean ?????
You have backups from many days ago. They are uncorrupted in the same way as other Comodo settings.

The flaw or forceful deletion could be some failsafe logic implemented in the parser. In this case you rules will disappear again (they will remain in backup).

That’s just a possibility.

 

[bazang]

He told everyone as early as 2011 that he will not pursue such methods (when it was times cheaper)

It's a long time ago but IIRC he started to say it on various parts of the Comodo forum soon after the original forum was launched. Back then, he had so many software projects running in parallel so he said it frequently.

user dependent modules like HIPS likely wouldn’t make it back. Not when Xcitium has the cloud verdict and the containment.

HIPS would not make it back because, while it took a very long time, software publishers learned that the only thing that works for people is full automation. Users are incapable of handling much of anything that requires a decision, no matter how well optimized the notifications or alerts are designed. Even IT Pros are discombobulated when it comes to decision making.

The popularity and revenues of Xcitium wouldn’t be enough for Xcitium to secure a loan given that it’s rather new legal entity as well. Relying just on revenue, they can’t invest more than 5-6% a year from already unimpressive numbers.

Melih's net worth is listed as $1 to $2 Billion USD, but - you know - he's very stubborn. I suppose one of his retorts to any criticisms would be "How do you think I got so rich? By being stubborn."

Everything that he's said that I've ever read leads me to believe he does not care nor want to capture maximum market share. What he wants is to prove that all the other players in the AV industry are dishonest with, if not outright scamming, everybody. The dishonesty part I understand because all corporations do it to one extent or another. But he does have a lot of valid points.

It should never have been designed in a way that one module would require a complete rewrite.
However other modules likely require rewriting for other reasons, Melih understands that and this is why he says a complete rewrite will be necessary.

With these facts, and the fact that Comodo has zero revenue and no dedicated product team or developers, it is extremely difficult to understand why anyone still complains about Comodo. I suppose they did not read Melih's posts from the early years and therefore do not have the context.

Or it could be just one messy overall platform that is not maintainable and scalable at all.

I bet the backend is even more hacky-hack, and it ain't cheap. I don't know why CAV was dropped, but I would not be surprised if the reason was the infrastructure expense.