Make your video test requests!

[ForgottenSeer 123526]

Thank you for these amazing ideas / insights! I was thinking more along the lines of tricky, as in just really good, unique samples that are difficult to render the correct verdict. But this is an interesting list of items we need to keep in mind as well, so thank you! I responded to each one to let you know the current state.

•Evasive Code: SiriusLLM should already perform well.

•"Living Off The Land" (LotL): This is more up to the cybersecurity software that SiriusLLM is integrated into. Although, thank you for mentioning this, you already have me thinking of some really cool ideas for the CyberLock, DefenderUI Pro and WDAC Lockdown integrations.

•Complex Obfuscation: SiriusLLM should already perform well.

Tricky False Positive Samples: SiriusLLM already performs extremely well with false positives. I have tested the hell out of it, with both benign and malicious samples, and I am simply astonished. Especially since this is the baseline and it is only going to get better from here.

•Benign Tools Mimicking Malware: Yeah, there is nothing we can really do about this. However, someone named Andy has suggested a couple of times that we auto block remote admin tools and let the user know that while it is a safe file, it is risky because others may have access to your system.

•Legitimate Installers/Updaters: If a legit file is tampered with, SiriusLLM will know and react accordingly. Having said that, we need to unpack files that contain other executables, scripts, etc, and analyze the contents.

•Obfuscated Benign Code: So far this has not been an issue. SiriusLLM has amazed me when it comes to this. Now, if a script is so obfuscated that it is unreadable, SiriusLLM is instructed to consider this in the verdict.

•User Scripts: SiriusLLM already performs extremely well. Do this... go to your favorite AI and ask them to create a funny .bat, .vbs, or whatever kind of script. And tell them to make it unusually long. Then analyze it with SiriusLLM and see the result . And it does not have to be a funny script, try any of the scripts you already have. BTW, while analyzing malware samples, there was this one Microsoft admin script that was super long, I believe it was a .vbs script downloaded from MalwareBazaar. To make a long story short, SiriusLLM spotted a few lines of code that were added to the standard Microsoft Script, that were malicious. It blew my mind.

Thank you so much for this detailed response. It's clear how much expertise and passion has gone into SiriusLLM, and your results are genuinely impressive. Hearing that it performs so well with evasive code and complex obfuscation reinforces exactly why an LLM is such a powerful tool in this space. The false positive rate you're seeing, even at baseline, is absolutely phenomenal, that's a massive differentiator.

 

[Shadowra]

Here’s a concise breakdown of challenging samples for testing @danb LLM, designed to highlight key areas of detection strength and weakness:

Tricky Malware Samples:

•Evasive Code: Malware using polymorphic/metamorphic code, heavy packing/encryption, or anti-analysis techniques (anti-VM, anti-debugging). The core malicious logic is hidden or constantly changing.

•"Living Off The Land" (LotL): Malicious use of legitimate system tools like PowerShell, WMIC, or Certutil. The tools themselves are benign; the danger lies in their contextual misuse.

•Complex Obfuscation: Code with convoluted control flow, junk instructions, hidden strings, or dynamic API calls. This makes raw code analysis difficult.

Tricky False Positive Samples:

•Benign Tools Mimicking Malware: Legitimate admin software (e.g., remote access tools, network scanners, pen-testing utilities) that perform actions similar to malicious activity (e.g., network connections, registry changes, process injection).

•Legitimate Installers/Updaters: Software that legitimately modifies system files, creates services, or downloads components, resembling malware installation.

•Obfuscated Benign Code: Lawful applications or scripts that use packing, compression, or intentional obfuscation (for IP protection) which might trigger generic suspicious patterns.

•User Scripts: Personal automation scripts that perform unusual system interactions (e.g., mass file operations, non-standard downloads) but are entirely benign in intent.

Focusing on these areas will provide a robust benchmark for SiriusLLM's ability to discern true malicious intent from complex or ambiguous code.

@danb
I'm going to use what he said and do a nice review.
That's why I'm delaying the video I was supposed to release on SiriusLLM, just to do something clean
(I'll put it all together early next week, until I can get it all ready)

 

[danb]

@danb
I'm going to use what he said and do a nice review.
That's why I'm delaying the video I was supposed to release on SiriusLLM, just to do something clean
(I'll put it all together early next week, until I can get it all ready)

Very cool, thank you, I appreciate your efforts!

 

[badboy]

Hi, Shadowra. Maybe you were interesting this Indian AV -- T9 on Avira engene.