CrowdStrike broke Debian and Rocky Linux months ago, but no one noticed

Gandalf_The_Grey

Level 85
Thread author
Verified
Honorary Member
Top Poster
Content Creator
Well-known
Forum Veteran
Apr 24, 2016
7,757
6
82,472
8,389
54
The Netherlands
A widespread Blue Screen of Death (BSOD) issue on Windows PCs disrupted operations across various sectors, notably impacting airlines, banks, and healthcare providers. The issue was caused by a problematic channel file delivered via an update from the popular cybersecurity service provider, CrowdStrike. CrowdStrike confirmed that this crash did not impact Mac or Linux PCs.

It turns out that similar problems have been occurring for months without much awareness, despite the fact that many may view this as an isolated incident. Users of Debian and Rocky Linux also experienced significant disruptions as a result of CrowdStrike updates, raising serious concerns about the company's software update and testing procedures. These occurrences highlight potential risks for customers who rely on their products daily.

In April, a CrowdStrike update caused all Debian Linux servers in a civic tech lab to crash simultaneously and refuse to boot. The update proved incompatible with the latest stable version of Debian, despite the specific Linux configuration being supposedly supported. The lab's IT team discovered that removing CrowdStrike allowed the machines to boot and reported the incident.

A team member involved in the incident expressed dissatisfaction with CrowdStrike's delayed response. It took them weeks to provide a root cause analysis after acknowledging the issue a day later. The analysis revealed that the Debian Linux configuration was not included in their test matrix.

"Crowdstrike's model seems to be 'we push software to your machines any time we want, whether or not it's urgent, without testing it'," lamented the team member.

This was not an isolated incident. CrowdStrike users also reported similar issues after upgrading to RockyLinux 9.4, with their servers crashing due to a kernel bug. Crowdstrike support acknowledged the issue, highlighting a pattern of inadequate testing and insufficient attention to compatibility issues across different operating systems.
 
They broke Linux, they broke Windows... They should brake MacOS next. It would be a shame not to have all the achievements when they are so close...
The funny thing is CrowdStrike Falcon is such a tiny client, installed, it doesn’t exceed 50MB. It merely serves as a sensor, capturing new files (subjected to emulation), new objects (named pipes, mutexes, registry keys) and system events. It’s not one of these over-complicated products that have massive room for error. Not sure how they managed to mess up so badly with such a simple client.
 
The funny thing is CrowdStrike Falcon is such a tiny client, installed, it doesn’t exceed 50MB. It merely serves as a sensor, capturing new files (subjected to emulation), new objects (named pipes, mutexes, registry keys) and system events. It’s not one of these over-complicated products that have massive room for error. Not sure how they managed to mess up so badly with such a simple client.
They obviously have an unmatched talent :)