This kind of evolved out of a few discussions (ESET, Emsisoft, etc) where the abilities of their behavior blocker came to question. As a part of this, I made two pieces of zero-day "malware". For now I'm avoiding their distribution because of all the legal implications of writing and distributing malware. The goal here was to come up with something that has ZERO known signatures... I compile it against a slightly different toolchain for each test and opted out of cloud submission as long as it does not affect detection.

The two samples are:
Sample 1: Simulated PUA/Replication: Copies itself to the temp directory under a brand new name. Registers that copy as a startup item via the registry.
Sample 2: Simulated Ransomware: Goes into "My Documents\test", and encrypts each file with a randomly generated AES256 key in a ".encrypted" file, deletes the original.

Result definitions:
Protected: Threat blocked with default settings
Protected (Folder Protection Only): For the ransomware sample, requires a specific anti-ransomware feature (like protected folders) to be turned on
Infected: For the startupware sample, AV suite had no reaction to it.
Partial: For the startupware sample, AV suite had a late reaction (such as after a reboot). This is overall acceptable since the sample has no malicious behavior, catching it after a reboot is perfectly fine.
Encrypted: For the ransomware sample, no matter what settings were tried, all files ended up encrypted.


SampleKaspersky AntivirusF-Secure SAFE 17.7Windows DefenderESET NOD32 13Norton 360Emsisoft Antimalware
Simulated PUA/ReplicatorPartial *ProtectedInfectedInfectedProtected + Partial Protected
Simulated RansomwareProtected *ProtectedProtected (Folder Protection Only)EncryptedEncryptedProtected

Overall Conclusion:
F-Secure, Emsisoft, and Kaspersky provided the best protection. Overall, F-Secure and sorta Emsisoft won by a thin margin, because of a bug I hit in Kaspersky (see below)

Additional Notes:
  • Kaspersky: Did a great job cleaning up the startup items and rolling back changes done. For the simulated ransomware, it reacted differently to each file being renamed (protected) vs just putting all the encrypted files into one password-protected ZIP file and then deleting all the files (no reaction). WARNING: Kaspersky's behavior blocker did not activate until after first reboot. Even though it appears to be functional after a fresh install, always reboot the machine after installing Kaspersky or you might not be protected. Since this only affects initial installation, I don't find it very concerning.
  • F-Secure: All detections came from DeepGuard. Sample 1 had a disappointingly generic detection name but Sample 2 performance was impressive -- even disabling "Ransomware" (protected folder) detection, DeepGuard still thinks it's ransomware behavior.
  • Windows Defender: Turning on the Ransomware Guard feature resulted in the only behavior block during this test. I should say, though, Windows Defender's ransomware protection feature is extremely sensitive and FP prone. For example, it also blocked me from running the Norton 360 installer because the installer creates a Norton Downloaded Files directory on the desktop.
  • ESET: As discussed thoroughly in the ESET thread, did not react at all. According to their forums, their HIPS is more for detection of variants of existing malware, not entirely new breeds.
  • Emsisoft: Did great against both tests with default settings. No complaints (other than a hang when running the binaries off the network)
  • Norton: SONAR picked up the startup sample right away but it cleaned up the original file as opposed to the replicated binary, SONAR picked it up again on reboot and this time it cleaned it up completely. No reaction against the ransomware simulator though.

I'm curious if this kind of testing is considered valuable. Happy to take suggestions of what other behaviors are common for in the wild malware samples and worthwhile of simulating. Remember that this is solely meant as a behavior blocker test, not an overall AV test. Behavior blocking is just one layer of protection that an AV provides.
 

Burrito

Level 21
Verified
I'm curious if this kind of testing is considered valuable. Happy to take suggestions of what other behaviors are common for in the wild malware samples and worthwhile of simulating. Remember that this is solely meant as a behavior blocker test, not an overall AV test. Behavior blocking is just one layer of protection that an AV provides.
I commend you on your test and sharing your test data with us in an easily read and informative narrative.

Although the sample set is tiny, it can still be informative as part of larger look at capabilities.

I encourage you to send the samples to @silversurfer so that they can be made part of the Hub testing process here at MT.

(y)
 
I encourage you to send the samples to @silversurfer so that they can be made part of the Hub testing process here at MT.
I will think about it -- my concern is that it's technically a federal crime to distribute malware and I don't have the legal expertise to know if what I'm doing can be construed that way.


Indeed, CFA is quite sensitive and implementation could be improved substantially. Thanks for your test.
Yeah I think the biggest things I'd like to see from WD's folder protection are:
(1) prompt instead of just flat out blocking and causing the application to error out
(2) have a mode that just protects deletion or modification of existing files, not the creating or reading of existing ones. Having creation of files be blocked seems kind of overkill.
 

blackice

Level 13
Verified
I will think about it -- my concern is that it's technically a federal crime to distribute malware and I don't have the legal expertise to know if what I'm doing can be construed that way.




Yeah I think the biggest things I'd like to see from WD's folder protection are:
(1) prompt instead of just flat out blocking and causing the application to error out
(2) have a mode that just protects deletion or modification of existing files, not the creating or reading of existing ones. Having creation of files be blocked seems kind of overkill.
The issue with a prompt is an average user will click through to access whatever file they were trying to open. Although I personally would also prefer a prompt. The endless whitelisting has driven me to turn CFA off.
 
The issue with a prompt is an average user will click through to access whatever file they were trying to open. Although I personally would also prefer a prompt. The endless whitelisting has driven me to turn CFA off.
Agreed! The latest macOS basically has this behavior built in by default. Anything not properly signed/endorsed by Apple will trigger a prompt if they try to touch certain things like documents. If they do it from an interactive session their app gets hung until the user makes a decision. If they do it from a background daemon it just gets rejected.

The criticism is that it results in a lot of potentially annoying prompts, but on the bright side, without an element of phishing it essentially protects against any sort of ransomware or espionageware
 

TRS-80

Level 1
G'day @MacDefender ,

Surely if you pack and password the files, sending the passwords separately you'd be fine. The samples are merely shared research. Most of that type of legislation has provisions for study, research & development and, so forth.

I'd be checking the legal definition of “distribute” if you're worried. I'd do it for you but I speak Aussie law.

I hope you manage to work around it!

Cheers,


@TRS-80 :emoji_beer:
 

notabot

Level 15
Thanks for sharing this ! - During your testing for WD was ASR rule "Block executable files from running unless they meet a prevalence, age, or trusted list criterion " on or off ?

I will think about it -- my concern is that it's technically a federal crime to distribute malware and I don't have the legal expertise to know if what I'm doing can be construed that way.




Yeah I think the biggest things I'd like to see from WD's folder protection are:
(1) prompt instead of just flat out blocking and causing the application to error out
(2) have a mode that just protects deletion or modification of existing files, not the creating or reading of existing ones. Having creation of files be blocked seems kind of overkill.
no lawyer will ever guarantee anything (because they can't) so not taking a legal risk is is probably the wise thing to do, and even if they did 300$ per hour for a domain expert's opinion is a steep price.

That said, I wonder what federal law says about source code, there's a precedent with distinguishing source from the artefact in the past with encryption (though this may not extrapolate to malware).
 
Thanks for sharing this ! - During your testing for WD was ASR rule "Block executable files from running unless they meet a prevalence, age, or trusted list criterion " on or off ?

WD was in the default settings initially, and then I turned on Controlled Folder Access after it failed both tests.

I expect the ASR rule you mentioned would block both executables but it would basically also block anything else I compile at my desk regardless of whether it's bad. I consider whitelisting a different feature independent of behavior blocking. It's fine if a behavior blocker uses prevalence to adjust its sensitivity but I think just blocking because it's not a popular binary should be tested separately.
 

notabot

Level 15
I expect the ASR rule you mentioned would block both executables but it would basically also block anything else I compile at my desk regardless of whether it's bad. I consider whitelisting a different feature independent of behavior blocking. It's fine if a behavior blocker uses prevalence to adjust its sensitivity but I think just blocking because it's not a popular binary should be tested separately.
True, see e.g. Q&A - ASR Rule: Block executable files from running unless they meet a prevalence, age, or trusted list criteria , you can add exceptions though ( I would had dropped the rule myself if this was not the case )
 
True, see e.g. Q&A - ASR Rule: Block executable files from running unless they meet a prevalence, age, or trusted list criteria , you can add exceptions though ( I would had dropped the rule myself if this was not the case )
Yep totally! I definitely think it's valuable and would love to test that on a different occasion.

This test was more meant as "assuming the worst case where someone tricked me into running something malicious, how well will my AV protect my data and identify malicious behavior happening in realtime?"

Another interesting test I think would be "how well can it clean unknown malware?" -- from this test it seems like Kaspersky would win here. It had the most comprehensive attempt to roll back a harmful chain of events.

F-Secure and Emsisoft both prevented the initial attempt to hook as a startup item. Norton's result was kind of funny because every time the system starts, Norton would realize THAT startup item was malicious and quarantine it, but it wouldn't realize that it had already replicated itself and registered another startup item for the next reboot 🤣. But infection detection and personal document protection are what I consider the most important. If my AV tells me it found something bad, it would trigger me to more carefully analyze my system. I do not need it to return my system to a 100% uninfected status.
 
One more observation: I would really wish that the suspicious behaviors would get presented to the user. Emsisoft shows “Suspicious/xxxx” in the logs and “xxxx” is like StartupItem or Ransomware and it is really intuitive. F-Secure is a mix — some DeepGuard signatures are named after a behavior, or after a specific kind of malware (seems like DeepGuard also uses behavior signatures to find variants of known malware) but other DeepGuard detections are just a jumble of lowercase letters.

Norton and Kaspersky are both really nondescript.

I suspect behavior blocker rules are part of their secret sauce and they hold these cards close to their chest. For example if you collect F-Secure diagnostic logs, they have a lot of plaintext logs — “Capricorn” is obviously an Avira scanner and has log entries when it consults the Avira cloud. “Lynx” is a certificate scanner and when executing a signed binary you get a log entry with the certificate it looks up and their results. But “HIPS” is DeepGuard and the log file is completely encrypted. It’s usually empty but after executing these two malware samples, it wrote out 8MB of random binary data to the HIPS log. Are they trying to thwart competitors or malware writers or both? Hmm I really wonder :)
 

Slyguy

Level 43
As a part of this, I made two pieces of zero-day "malware". For now I'm avoiding their distribution because of all the legal implications of writing and distributing malware.
This is true. Wise move. Slight tangent on this;

Remember, in the USA, you can basically do 'almost' anything you want in your home, and on your property. It's about action and intent. Also information sharing, even about dangerous things is protected by the constitution. There have been efforts by our govt. and/or factions within it to erode those protections and they are working hard to brainwash younger, upcoming generations to not know about and in most cases feel scared to exercise their freedoms. Cody Wilson is a good example.. He releases schematics for a working 3D Printed Gun, the Govt. hits him hard, actually violating the constitution in the process. Cody wins, so they honeypot him and toss him in jail for almost a decade. Cody's mistake was, underestimating the forces levied against him and thinking he was in the clear after embarrassing the government. Nah, active surveillance of him never stopped and they wanted revenge.

So yes sir, enjoy your malware, in your home, without intent to distribute and there cannot be any legal case against you at all. The same way I enjoy buying and giving away vintage copies of The Anarchists Cookbook with the constitution tucked into the front fold. :ROFLMAO:
 
Last edited:
So yes sir, enjoy your malware, in your home, without intent to distribute and there cannot be any legal case against you at all.
Thank you! This was very helpful! Unfortunately yeah my experience as a child in high school was my last attempt to just show a proof of concept exploit (responsibly) to the staff led to nearly getting the book thrown at me. I’d rather not chance it at this point — as an adult I’ve got a lot more to lose and a lot less spare time.

I will say, though, that I basically constructed these pieces of POC malware by typing each sentence from my description into StackOverflow and largely copy pasting :D. I’d be happy to elaborate more on describing what the ~10 lines of code do line by line for anyone else to replicate.