Advice Request Cross-Platform Hash Comparison to Detect Ransomware - Backup Software Hole?

[raywood]

I have developed a scheme to detect a type of ransomware. It seems that my scheme may have a hole.

The general idea is that I calculate SHA-512 values for each file on a source data drive; I compare those values against SHA-512 values for each file on a backup drive; and I review discrepancies to see whether they are highlighting ransomware activity. In theory, this should detect the type of ransomware that silently encrypts data files, decrypting them on the fly, on demand, so that the user will have no notice that files on the source drive are gradually being encrypted.

I am told that such ransomware may prevent the operating system from recognizing that encrypted files have changed. But if I calculate the source drive hashes on a Windows system, and calculate the backup drive hashes on a Linux system, then (assuming the malware is not cross-platform) I should see discrepancies in hash values. The ransomware may fool the Windows OS, but the Linux hashing tool will hash the file as altered by ransomware, not merely its decrypted contents.

The problem is that, to update the backup drive, I am using software that detects file changes as indicated by timestamps and file sizes. I assume that (perhaps with the aid of compression) ransomware could manipulate both of those indicators, so that my backup software would see no change, and therefore would not reflect the actual condition of ransomware-encrypted files on the source (Windows) drive. I assume that a byte-for-byte comparison would detect the fact of change, but such comparisons are very slow, discouraging frequent use.

Am I wrong about any of this? I like the idea of being able to compare hash values to detect ransomware, but I'm afraid I may be going to a lot of trouble to achieve nothing.

 

[MacDefender]

As others have pointed out, the biggest flaw with this is that you are going to be asking the host OS for the contents of the files in order to compute a hash of them. If, in theory, this kind of malware is transparently encrypting and decrypting your data for you, it will probably do the same for your verification software so you won’t know the underlying files are encrypted.

BitLocker and other legitimate encryption software also will make an offline version of this difficult to achieve.

Not to mention, what about files that are actually meant to change? Nobody has all static documents, otherwise you might as well burn the contents to a physically read only disc and fire all your employees. How will such software determine what changes are and aren’t allowed?

Existing ransomware protection is already robust in terms of it being basically impossible to access or change a file in Windows without it triggering an antivirus API hook informing you. The weakness is that often times malware hides as a legitimate app, for example, using Excel macros to encrypt your spreadsheets so that to anti malware it looks like a legitimate user modifying spreadsheets

 

[raywood]

The comparison against hashes from the Linux system seems to address your first point.

What issue do you see with BitLocker - or, in my case, VeraCrypt? Hashing seems to work OK on both systems.

The comparison narrows down the vast pool of files to the few that have changed. My spreadsheet can sort those by folder, filename, and last modified date. So far, those criteria have enabled me to dismiss the vast majority of discrepancies between the Windows and Linux lists within a minute or so. Getting to that point takes longer. I haven't timed it exactly, but I think maybe 20 minutes. So there's no problem with allowing files to change.

I'm hoping Bitdefender will catch ransomware as you say. The reports I encountered seemed to indicate that the ransomware writers were pretty good at avoiding ransomware protection software.

 

[MacDefender]

The comparison against hashes from the Linux system seems to address your first point.

What issue do you see with BitLocker - or, in my case, VeraCrypt? Hashing seems to work OK on both systems.

The problem is you can’t easily mount those encrypted volumes from Linux. BitLocker uses the TPM and the TPM keys are derived from secure boot measurements of the windows boot loader and kernel. If you boot into a different OS like Linux, the TPM is unable to give you the OS specific key and you cannot unlock the volume.

On top of that, this would require a long interruption to the user’s activity to boot into a different OS and most of the times this wouldn’t be well tolerated.

The attacker can still use features like volume shadow copies and NTFS junction points to create encrypted copies of all your files or upload them to a remote server, such that you don’t see any on disk changes that matter, and then suddenly delete all of your original files one day.

The comparison narrows down the vast pool of files to the few that have changed. My spreadsheet can sort those by folder, filename, and last modified date. So far, those criteria have enabled me to dismiss the vast majority of discrepancies between the Windows and Linux lists within a minute or so. Getting to that point takes longer. I haven't timed it exactly, but I think maybe 20 minutes. So there's no problem with allowing files to change.

Well how do you scale this to, say, a company of Microsoft’s size? Are you hiring humans to look at modified files in backups? What about formats like SQLite databases or health records which are harder to inspect for changes?

I'm hoping Bitdefender will catch ransomware as you say. The reports I encountered seemed to indicate that the ransomware writers were pretty good at avoiding ransomware protection software.

Ransomware operators are good at avoiding protection mechanisms that their targets use. It’s not that the software is losing, it’s that all of these techniques are challenged when a group of 500+ qualified engineers in Russia are hell bent on compromising your data.

There’s nothing that would stop ransomware from disabling or modifying your software from verifying backups or subtly creating a bunch of false positives by adding a ton of harmless changes until your verifier gets exhausted of false positives and then it deletes the original files.

The biggest problem is not that ransomware is somehow impossible to detect or stop. It’s more that the cost and effort budget of real world IT departments is quite limited and over time, everyone lets their guard down. Most postmortems of big ransomware breaches have shown that there were obvious and preventable problems that led to the attack (particularly failing to patch exploitable vulnerabilities in a reasonable amount of time)

 

[raywood]

OK, sounds like a BitLocker issue. As I say, I use VeraCrypt.

They can upload my files if they want. There's nothing of interest to third parties there. I'll restore from backup. I'm concerned with malware that could corrupt backup.

I'm the user. I do the hashing on a secondary computer. It's old and not in demand. It can take its sweet time.

I'm not sure what to say about scaling up to Microsoft's size. I guess they have their problems too. I'm talking about my files.