Are hybrid-analysis reports trustworthy?

Status
Not open for further replies.

ZevinZenph

Level 1
Thread author
Verified
Mar 10, 2015
30
Recently I found that hybrid-analysis results are required (although you can just leave a "-" in the blank) while uploading malware samples to this forum. It's a pretty good web-based free analysis service with detailed reports at first glance.
3DEfR5N.png


But after uploaded a installer to this site, I'm now doubting the rating system used by this service is not as accurate as we think.

The file I uploaded is called "junior-icon-editor.exe", which is a free windows icon editor by sibcode and can be easily found on Google.

hxxp://www.sibcode.com/junior-icon-editor/

And here's its VT analysis report:
https://www.virustotal.com/en/file/...87aa2d62eca22ab61ef504fec9860918613/analysis/

As you can see, virustotal suggested that this file is likely to be harmless. (I've tried it with sandboxie and wireshark, found nothing suspicious with it.)

But the hybrid-analysis report tells a totally opposite result. This file is rated "malicious" based on their analyzing engine ,and "Sample was identified as malicious by at least one Antivirus engine."
https://www.hybrid-analysis.com/sam...62eca22ab61ef504fec9860918613?environmentId=2
xEa4LzJ.png


After seeing the result, I decided to do an experiment with a self-extracting zip file dropping a dummy .exe file (coded by myself using Dev-Cpp 4.9.9.2, and what it does is just asking you to insert some integers into the console screen), then execute the dropped file.

Here's the virustotal report of the zip file:
https://www.virustotal.com/en/file/...270648b01f55bf41a140dba9/analysis/1429368721/

And here's the hybrid-analysis report:
https://www.hybrid-analysis.com/sam...9c08a270648b01f55bf41a140dba9?environmentId=2

Thus, I don't think hybrid-analysis is a well-developed service and it's maybe not a good idea to keep the report as "required" since it may causes false positive on our forum.

Tell me your thoughts below! Any feedback are welcomed! :>
 

ZevinZenph

Level 1
Thread author
Verified
Mar 10, 2015
30
BTW, Sorry for typos. I'm not a native English speaker and I'm not able to edit the post because errors. :<
 

Payload Security

New Member
Nov 25, 2014
3
Hello ZevinZenph,

I'm the main developer of the service and saw this thread by accident. Wanted to clarify some points for you so you can understand what the service is about:

- Although the service shows a simplified classification such as 'malicious', 'suspicious' and 'no threat', you cannot read the reports as a result of an Anti-Virus engine, which has extremely high restrictions regarding false-positives/-negatives. When you read 'malicious', then you should read it as 'shows malicious behavior'. When you want to know if a sample is probably a malware sample, then you can e.g. read the 'Threat Score' (it's a value between 0/100 displayed beneath the banner; in your prime example it's 22/100, which isn't very high), but the bottom line is that you have to make a decision yourself reading the report. Right now any report that has only a single malicious behavior signature match automatically default to 'malicious' level, regardless of the threat score (we might change that in the future and require a threshold of the threat score), but since it's a forensic tool and not an anti-virus tool, that is okay for us.

- A lot of installers show a lot of behavior that is also typical for malware, e.g. creating child processes, dropping files on system pathways, connecting to a server to download additional files, maybe even using packers to have smaller executables, and so on. Thus, you will often find that installers uploaded to the service might receive a higher threat score due to all kind of malicious behavior being shown. The bottom line is: even if you would view the report not as a forensic report, but an anti-virus recommendation, then our motivation is to have an extremely low false-negative ratio (because that might cause a lot of damage), rather than a low false-positive ratio. But, like I said, it is the false condition under which you are reading the report.

Here is a good example that outlines what I said:

1) This Bartalex analysis had a 0/57 at VT but dropped some files that in turn matched on VT: https://www.hybrid-analysis.com/sam...74f88c86d0423054643ac1639f76e?environmentId=1

... if we take a look at the report itself we can quickly see what's going on: malicious macro is executing, surfing to pastebin.com to find a C2 URL to drop the next stage malware. Connecting directly to 95.163.121.201 via a hardcoded IP (i.e. no prior DNS lookup) is another indicator of maliciousness. I think anyone looking at the report will probably come to the conclusion that one would rather not launch the file. Neither the VT results nor the Threat Score of 74/100 really can give an advisory, the manual task of checking the report details is mandatory.

2) This is an opposite example of GPU-Z, which is a known tool to display information about the system. If you check out the report: https://www.hybrid-analysis.com/sam...1c060ca710598187c0039aea6ce63?environmentId=1

... then you will quickly see a lot of behavior that is typically found in malware: a system driver is being dropped, it has the ability to reboot your system actually, it's packed with PECompact, creates child processes, and so on. The only reason anyone would launch the file is because we know the authors are trustworthy and the file might be on some larger whitelist. As there is no whitelist lookups, we classify the sample as 'malicious' and give it a 85/100 Threat Score actually, as it has a lot of artifacts that are typical for malicious files. Again: it is mandatory for you to look into the file and make your decision based upon your experience or additional lookups (e.g. you could look up techpowerup.com and find that it is a valid domain).

Hope that helped get the most out of the service.

Update: we just changed the classification and require at least a threat score of 50/100 for the report to be classified as 'malicious' (but the explanations above apply nevertheless). In your case though, the report is classified as 'suspicious', because the threat score is 22/100.
Update #2: funny enough, your dummy.exe is actually classified by ~4 AVs as malicious, including Symantec.
 
Last edited:

Payload Security

New Member
Nov 25, 2014
3
Uploaded on of my worm removal tool and here is its verdict: Malware

https://www.hybrid-analysis.com/sam...0dc2bfb382f4e302c5926c937a3d0?environmentId=2

Actually with our new threshold of 50/100 it is classified as 'suspicious' now, because the threat score is only 24/100, but regardless of that cosmetic change, you need to read it as 'shows behavior that is malicious' or 'shows behavior that is suspicious', the 'threat score' is just the degree. See my explanations below, it's not an AV tool, it's a forensics tool.
 

WinXPert

Level 25
Verified
Honorary Member
Top Poster
Malware Hunter
Well-known
Jan 9, 2013
1,457
Actually with our new threshold of 50/100 it is classified as 'suspicious' now, because the threat score is only 24/100, but regardless of that cosmetic change, you need to read it as 'shows behavior that is malicious' or 'shows behavior that is suspicious', the 'threat score' is just the degree. See my explanations below, it's not an AV tool, it's a forensics tool.

Yeah a forensic tool with a bad verdict
 
Status
Not open for further replies.

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top