Advice Request Engines on VirusTotal, worse version?

Please provide comments and solutions that are helpful to the author of this topic.

ShenguiTurmi

Level 3
Thread author
Well-known
Feb 28, 2023
126
I just got a new backdoor sample from someone and it will inject a lot of (random?) processes and start payloads, and I found that DeepInstinct detected it when I tested it.
di1.png
Then I uploaded it to VT to see if other security software had detected it, and that's when a very strange scene occurred.
di2.png
As shown in the image, DeepInstinct on VirusTotal does not detect it.
I initially thought that it was the very high threshold of ML's confidence on VT, which I set to a medium threshold, that produced the result, but I observed that it was not so simple.
di3.png
Yes, in DeepInstinct's backend I found that the machine learning gives a very high degree of confidence, meaning that the sample should be detected at any setting. This is very strange, and the only reason is that VT and I do not have the same ML engine or model.
I asked other friends who follow security and I was told that TrendMicro often has different results than VirusTotal when they test as well.
I understand that there may be some different results on VirusTotal due to limitations, such as Avira and Gridinsoft dont have cloud. But I was very surprised that even the basic ML results were different.

Have you guys noticed such phenomenon? Or is this just an isolated case that I am experiencing?
It may be inappropriate to post virus samples in the public section, so I'll just include the link to VT: VirusTotal
 

Kongo

Level 36
Verified
Top Poster
Well-known
Feb 25, 2017
2,527
I just got a new backdoor sample from someone and it will inject a lot of (random?) processes and start payloads, and I found that DeepInstinct detected it when I tested it.
Then I uploaded it to VT to see if other security software had detected it, and that's when a very strange scene occurred.
As shown in the image, DeepInstinct on VirusTotal does not detect it.
I initially thought that it was the very high threshold of ML's confidence on VT, which I set to a medium threshold, that produced the result, but I observed that it was not so simple.
Yes, in DeepInstinct's backend I found that the machine learning gives a very high degree of confidence, meaning that the sample should be detected at any setting. This is very strange, and the only reason is that VT and I do not have the same ML engine or model.
I asked other friends who follow security and I was told that TrendMicro often has different results than VirusTotal when they test as well.
I understand that there may be some different results on VirusTotal due to limitations, such as Avira and Gridinsoft dont have cloud. But I was very surprised that even the basic ML results were different.

Have you guys noticed such phenomenon? Or is this just an isolated case that I am experiencing?
It may be inappropriate to post virus samples in the public section, so I'll just include the link to VT: VirusTotal
Did DeepInstinct detect it pre or post execution? In your screenshot it says that the Deep Static Analysis detected the sample. Normally the static detection of the product, should also be available on VirusTotal, but if the detection was dynamic and after execution, then it's normal that VirusTotal doesn't catch it.
 

struppigel

Moderator
Verified
Staff Member
Well-known
Apr 9, 2020
661
From what I read on their website, DeepInstinct is solely based on static analysis.You are right that in this case the expectation would be that VT results are the same.

Here are a number of reason I can think of, why the result still differs, but all of these are speculation!
  • Updates of the AI model can arrive later on VT than on endpoints, so yours might just be more up-to-date
  • DeepInstinct might also include contextual data, e.g., where the file was downloaded from, or if it was downloaded at all, where it is currently on the filesystem, etc. That data will not be there or will be different on VT
  • The settings might be different on VT. You said the confidence indicates that all settings should have detected this threat, but who knows what else those settings entail. It can be more complex than just confidence level adjustment
If you are really curious, ask DeepInstinct support.
 

harlan4096

Moderator
Verified
Staff Member
Malware Hunter
Well-known
Apr 28, 2015
8,728
This is not new, and also happens with other products, confirmed here with Kaspersky, and others security products I've been testing in Malware Hub of the forum, so VT results/detections must be taken with "a sack of salt" 😁
 

ShenguiTurmi

Level 3
Thread author
Well-known
Feb 28, 2023
126
Did DeepInstinct detect it pre or post execution? In your screenshot it says that the Deep Static Analysis detected the sample. Normally the static detection of the product, should also be available on VirusTotal, but if the detection was dynamic and after execution, then it's normal that VirusTotal doesn't catch it.
pre execution.
That's why I'm confused......
 

Bot

AI-powered Bot
Apr 21, 2016
3,641
I just got a new backdoor sample from someone and it will inject a lot of (random?) processes and start payloads, and I found that DeepInstinct detected it when I tested it.
Then I uploaded it to VT to see if other security software had detected it, and that's when a very strange scene occurred.
As shown in the image, DeepInstinct on VirusTotal does not detect it.
I initially thought that it was the very high threshold of ML's confidence on VT, which I set to a medium threshold, that produced the result, but I observed that it was not so simple.
Yes, in DeepInstinct's backend I found that the machine learning gives a very high degree of confidence, meaning that the sample should be detected at any setting. This is very strange, and the only reason is that VT and I do not have the same ML engine or model.
I asked other friends who follow security and I was told that TrendMicro often has different results than VirusTotal when they test as well.
I understand that there may be some different results on VirusTotal due to limitations, such as Avira and Gridinsoft dont have cloud. But I was very surprised that even the basic ML results were different.

Have you guys noticed such phenomenon? Or is this just an isolated case that I am experiencing?
It may be inappropriate to post virus samples in the public section, so I'll just include the link to VT: VirusTotal
It's not uncommon for different security vendors to have different detection rates when it comes to virus and malware samples. This is because each vendor has their own unique set of ML models, heuristics, and algorithms that they use to analyze and detect threats. In addition, not all vendors have access to the same data sources or intelligence feeds, which can also impact detection rates.

Furthermore, it's important to note that VirusTotal is just a tool that aggregates results from various security vendors, and as such can only provide a snapshot of the detection capabilities of those vendors at a specific point in time. It's possible that detection rates for a particular sample may change over time as vendors update their ML models and heuristics.

In your specific case, it's hard to say for certain why DeepInstinct and VirusTotal had different results for that backdoor sample. That being said, it's not uncommon for ML models to have different detection rates depending on the data they are trained on, the features they use, and the algorithms they employ. So it's entirely possible that DeepInstinct and VirusTotal simply have different ML models that are more or less effective at detecting that particular sample.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top