Tutorial How Dynamic Heuristics Analysis Works [REVISED]

W

Wave

Guest
#1
Hello everyone. :)

This thread is a new revision of my old thread: How-to Guide - How Dynamic Heuristic Analysis Works - hopefully this version will be much more beneficial.


----------------------------------------------------------------------------------------------------------------------------
It’s recommended that you read the following threads prior to reading this one to obtain a better understanding on specific references used within this thread (read the theory in the below threads):

https://malwaretips.com/threads/windows-built-in-protection-mechanisms.66165/
https://malwaretips.com/threads/theory-c-tutorial-native-windows-api-ntapi.63573/
https://malwaretips.com/threads/developing-your-own-anti-exe-c-device-driver-development.64043/
https://malwaretips.com/threads/critical-processes-theory-c-c-vb.67002/
----------------------------------------------------------------------------------------------------------------------------

Today I will be discussing how dynamic heuristics works and how it can be implemented – please remember that it can be approached through many different ways and everyone’s thinking is different, therefore not all security products which include dynamic heuristics may work the same. This is neither going to be a tutorial for developing dynamic heuristics, only theory-based.

When it comes to dynamic heuristics there are many different stages for implementation, I will start off by explaining the definition of dynamic heuristics and then I will proceed to discuss the internals behind the implementation of the component.

Definition of dynamic heuristics
Dynamic heuristics is essentially heuristic analysis of a program while it is executing in memory (hence the word “dynamic”); this means that the running program is being monitored and information about how it’s working is being logged and this logged information is being assessed to determine if the program is suspicious/malicious or safe. Whereas, static heuristics, is scanning of the program while it is not running in memory (based on its normal characteristics prior to being executed).

How dynamic heuristics could be implemented
When it comes to development of dynamic heuristics there are numerous approaches and you’ll need to focus your mind-set onto more than one specific thing; as we already know, there are thousands of new threats released into the wild every week, therefore the dynamic heuristics needs to be tuned to detect new threats just as well as the old ones (if new threats weren’t being picked up then it’d be pretty pointless to implement the feature, since we can obtain old malware and add a detection via checksum hashes or generic detection methods via static heuristics). This means that we need to make sure we do proper research on numerous different types of malware threats and implement some sort of identification system based on the behaviour for as many malware types/variants as we can. Throughout this section I will briefly discuss how we could implement identification toward specific malware types, however I will also discuss how we could implement identification for general suspicious programs which cannot be matched to a specific variant (therefore it’d just be labelled as a Trojan most likely – a Trojan is where a program is malicious however it pretends to be something it isn’t).

To get started, we need to ensure that we are monitoring the program’s behaviour from the moment it starts executing its own code, which means we need to be on-watch for all new process start-up attempts. To monitor for all new process start-up attempts we need to decide on a reliable and stable method which cannot be easily bypassed, and the solution to this would be through using a documented kernel-mode callback which happens to be called PsSetCreateProcessNotifyRoutineEx. The downside to using a kernel-mode callback is that we will be required to develop a device driver and this may not be a suitable option for everyone since to load a device driver on a 64-bit system we will require to have applied code signing authentication (which means we need to digitally sign the device driver *.sys binary), and obtaining a digital signature can be pricey depending on the scenario – if you happen to find yourself in a situation where you need to monitor process execution which is system-wide entirely from user-mode, one of your best bets in terms of performance would be to work with API hooking methods, since constantly enumerating through the running processes will cause high CPU usage and will not be able to intercept before the program’s code can actually start running. If you ever find yourself in a situation where you are setting API hooks to monitor process creation, as long as you are not focusing support on Windows XP, you can hook NtCreateUserProcess which is exported by ntdll.dll; to bring back Windows XP support, hook RtlCreateUserProcess instead.

After we have successfully setup process monitoring in a way that allows us to perform an action before the main thread of the process is resumed and actually starts executing the program’s code, we need to brainstorm how we are going to monitor the program’s behaviour – we need to log which API functions it calls which can allow us to discover a pattern in behaviour based on the APIs called in a sequence. To do this, we will need to hook the APIs within the scope of our monitoring so we can log these API calls before they are actually called – a common method would be via IAT (Import Address Table) hooking. However, before we hook APIs, we need to get our own code executing within the address space of the process which is starting-up. We can make use of a Dynamic Link Library (DLL) to hold our code and then inject this into the process, or we can work with more stealth techniques, which would be code injection without requiring any dependencies (meaning we allow our code to be executed within the process however no DLL is required).

The most common and easiest method of DLL injection would be through using Win32 API functions: VirtualAllocEx, WriteProcessMemory and CreateRemoteThread. A more sophisticated version of this method would be through using the Native API equivalents (in order of the functions I just listed, it would be: NtAllocateVirtualMemory, NtWriteVirtualMemory and RtlCreateUserThread/NtCreateThreadEx). This method works by allocating memory in the target process (which will be used to store the file path of our DLL to be injected), then we will write to that allocated memory to pass in the file path of our DLL into that newly allocated memory, and then we create the remote thread (however when we create the remote thread we pass in LoadLibraryA and the variable holding the DLL path as the parameter – this causes it to call LoadLibraryA(DLLPATH) which means the process will then load our DLL into memory). The method is not very stealth and a better alternative would be manual mapping injection however this won’t be covered in this thread.

Once you’ve successfully setup monitoring for process execution and got your injection method sorted out, now you can focus on what functions you should be hooking and what the purpose of hooking them would be – there are numerous approaches when it comes to dynamic heuristics for determining when you should automatically intercept and block an action, or at worst case scenario, completely quarantine the program running in memory based on its behaviour attempts. A common method would be to work with a scoring system (e.g. every-time an API call is made and the filtering in the call back proves this call to be suspicious/malicious, you could increment the scoring system) and once the score is above a specific value then you could automatically intervene and quarantine the sample (and even give it a matched detection name if possible based on the logged behaviour – however this may not always be reliable nor possible). As an additional assessor to this, you can watch out for actions where the chances of them being malicious than not are high, and then automatically block this action (and alert the user as well), but not quarantine the program until the scoring system hits its peak.

Before I make a list of functions which could potentially be hooked to monitor certain actions/behaviour, I will start by listing some functions which are commonly found in specific threat types.

Trojan Keylogger: keylogger’s are famous for logging the keystrokes typed by the user through setting hooks on the keyboard via the Win32 API. Within the Windows API there is a function known as SetWindowsHookEx and this can be used to set system-wide hooks, allowing you to receive a notification on the registered events. SetWindowsHookEx isn’t always used for keylogging though, it has many genuine uses, however it’s rare to find normal programs to use it manually for a genuine purpose… Naturally, Windows programs tend to use it when the window is dragged (if the feature is enabled, if you move a window really quickly by the drag of the title area, it’ll minimize all other windows behind it – this is done through the usage of SetWindowsHookEx). The function SetWindowsHookEx is exported by User32.dll and the Native API equivalent is NtUserSetWindowsHookEx (not accessible from user-mode – NtUserSetWindowsHookEx is exported by win32k.sys). The function takes in 4 parameters, however the first one is the only one we are interested in right now and it’s data type is of an integer – the supported values for the first parameter are as follows: WH_MSGFILTER (-1); WH_JOURNALRECORD (0); WH_JOURNALPLAYBACK (1); WH_KEYBOARD (2); WH_GETMESSAGE (3); WH_CALLWNDPROC (4); WH_SYSMSGFILTER (6); WH_MOUSE (7); WH_DEBUG (9); WH_SHELL (10); WH_FOREGROUNDIDLE (11); WH_CALLWNDPROCRET (12); WH_KEYBOARD_LL (13); and WH_MOUSE_LL (14).

If you hook SetWindowsHookExW/A for preventing hooks on the keyboard being set, make sure to filter out the first parameter in the callback function – you’ll need to make sure that it really is being set for the keyboard, therefore you can compare the value (integer) which is passed as the first parameter to WH_KEYBOARD and WH_KEYBOARD_LL. On top of this, you can watch for the usage of the function GetAsyncKeyState which is exported by User32.dll as well.

Trojan Rootkit: Rootkits are designed to conceal evidence of a malware infection which is present on the system and therefore they tend to be working from a “root” level (e.g. they have full control to do what they need to do). The term “rootkit” actually comes from Linux (hence “root”), however a rootkit is basically a set of tools for concealment/protection of an existing malware infection on the system. If you look online for other definitions then it may be different, however based on my own experience with rootkit development and analysis, they’re a set of tools used to conceal and protect malware – therefore they’d be pointless on their own, because they’d have nothing to conceal/protect, however in more sophisticated malware there may be a rootkit component so it can evade detection from the user and potentially any security products installed on the system also. Rootkits do not always have full control like they are supposed to, sometimes they are limited but still get the job done to a reasonable extent… An example of this would be a user-mode rootkit, where it’s execution is limited to user-mode and therefore it can be identified by code being executed at a kernel-level (higher privileges than it has).

When it comes to dynamic detection of rootkits it can be a very tricky business since rootkit technology is forever evolving (malware generally-speaking is evolving constantly) and therefore it is pretty much impossible to detect everything… Although you can do your best by implementing identification techniques to detect rootkits using methods which are already known to us.

The first thing you can do in terms of rootkit identification is to watch for injection attacks; this means you will also have dynamic heuristics for “Trojan Injector” (as some vendors label an injector in the detection names). You can do this through hooking a bunch of APIs which are associated with code injection, and then determining when the APIs are really being used for an injection attempt (since the APIs used for malicious injection attacks can be used manually for genuine purposes, but are also naturally used by the Windows OS itself within the program’s address space for normal functionality. E.g. if you use the Task Scheduler then your program will attempt to call NtAllocateVirtualMemory where the process handle is for schtasks.exe however you wouldn’t necessarily know about this unless you hooked the function and then tested it to see if the hook was triggered for when schtasks.exe is operated with). Identifying injection attacks without causing false positive detections can be difficult depending on how you approach the situation. A good bet would be focusing on functions like NtWriteVirtualMemory and RtlCreateUserThread/NtCreateThreadEx – if the program calls NtWriteVirtualMemory then log the Process ID it is attempting to write memory too, and then if it calls an NTAPI function for remote thread creation then you can flag this as a malicious injection attack, however extensive filtering can go a long way to preventing false positive alerts.

The second thing you can do in terms of rootkit identification is to scan for rootkit-related modifications; this can include (but is not limited to) SSDT (System Service Dispatch Table), IAT (Import Address Table), EAT (Export Address Table) hooking and DKOM (Direct Kernel Object Manipulation). To scan the SSDT and identify DKOM you’ll need to load a device driver to get your code executing in the space of kernel-mode, however you can scan for user-mode rootkit techniques from user-mode itself (using system calls would be a good idea since they are capable of bypassing user-mode hooking mechanisms).

For injection attacks, you can monitor the usage of the following functions:
Code:
ntdll.dll!NtReadVirtualMemory
ntdll.dll!NtAllocateVirtualMemory
ntdll.dll!NtWriteVirtualMemory
ntdll.dll!NtCreateThreadEx
ntdll.dll!RtlCreateUserThread
ntdll.dll!NtQueueApcThread
ntdll.dll!NtSuspendThread
ntdll.dll!NtResumeThread
ntdll.dll!UnmapViewOfSection
kernel32.dll!GetThreadContext
kernel32.dll!SetThreadContext
If you are capable of hooking the above functions and monitor the usage that’ll be more than enough to not only block DLL injection but also other stealth methods such as hijacking of a process’ thread/s.

Moving away from specific threat types, I’m going to list a bunch of functions for monitoring specific activity (as long as they are hooked). Personally, I think that a device driver with a callback to CmRegisterCallbackEx and also a mini-filter filesystem driver is more secure and beneficial than API hooking for monitoring registry and I/O instructions, however it’s really down to how you want to design your system and how much time you have since the latter will be more time consuming to develop, and will require additional resources.


I/O operations:
Code:
NtReadFile
NtCreateFile
NtWriteFile
NtSetInformationFile
NtQueryInformationFile
All the above listed functions are exported by ntdll.dll (Native API).

Registry operations:
Code:
NtOpenKey
NtQueryKey
NtQueryValueKey
NtQueryMultipleValueKey
NtCreateKey
NtDeleteKey
NtSetValueKey
NtReplaceKey
NtEnumerateKey
NtEnumerateValueKey
NtSetInformationKey
NtUnloadKey
All the above listed functions are exported by ntdll.dll (Native API).

Other functions which you can monitor which may or may not be exclusive to specific operations:
Code:
ntdll.dll!NtCreateUserProcess
ntdll.dll!RtlCreateUserProcess
ntdll.dll!NtOpenProcess
ntdll.dll!NtTerminateProcess
ntdll.dll!NtSuspendProcess
ntdll.dll!NtOpenThread
ntdll.dll!NtTerminateThread
ntdll.dll!RtlSetProcessIsCritical
ntdll.dll!NtProtectVirtualMemory
ntdll.dll!NtQueryVirtualMemory
ntdll.dll!NtLockVirtualMemory
ntdll.dll!NtUnlockVirtualMemory
ntdll.dll!NtFreeVirtualMemory
ntdll.dll!NtShutdownSystem
ntdll.dll!LdrLoadDll
ntdll.dll!LdrGetProcedureAddress
ntdll.dll!LdrUnloadDll
ntdll.dll!NtQuerySystemTime
ntdll.dll!NtSetSystemTime
ntdll.dll!NtSetSystemInformation
ntdll.dll!NtQuerySystemInformation
ntdll.dll!NtLockFile
ntdll.dll!NtUnlockFile
ntdll.dll!NtDeviceIoControlFile
ntdll.dll!NtCreateNamedPipeFile
ntdll.dll!NtLoadDriver
ntdll.dll!NtUnloadDriver
ntdll.dll!NtSetSystemInformation
ntdll.dll!NtCreateEvent
ntdll.dll!NtSetEvent
ntdll.dll!NtDebugActiveProcess
ntdll.dll!NtSetSecurityObject
ntdll.dll!NtQuerySecurityObject
ntdll.dll!NtCreateSection
ntdll.dll!NtOpenSection
ntdll.dll!NtUnmapViewOfSection
sechost.dll!CreateServiceA
sechost.dll!CreateServiceW
sechost.dll!StartServiceA
sechost.dll!StartServiceW
user32.dll!SetWindowsHookExA
user32.dll!SetWindowsHookExW
user32.dll!SetWinEventHook
There are tons more of functions which can be monitored to increase the scope of what is monitored/protected against, however your best bet is to go through open source malware source code and reverse existing samples in the wild to understand how they work and which functions they call to function; then you can work with a debugger to get the NTAPI equivalent from the Win32 API functions being used (target the NTDLL functions if possible). Another idea is to open up Windows DLL files like ntdll.dll, kernel32.dll, user32.dll, etc. in IDA Pro and check the Imports so you know which functions are embedded within the DLL; then you can perform research on functions which seem interesting to you – bear in mind that not all functions are documented by Microsoft themselves, or even anyone potentially.

--------------------------------------------------
If you want to take things up a step to make sure everything is secure, then I suggest you research usage of the hyper-visor/Intel VT-x & AMD SVM for real virtualization – if you can successfully implement usage of these, then you can virtualize the programs to be monitored which makes things much more secure; you’ll be able to forget about PatchGuard/Kernel Patch Protection and make use of kernel-mode patching such as MSR hooks on x64 systems, also. The only vendor which has done such a thing for the overall system so far would be Kaspersky I believe, however Comodo will also work with real virtualization for their auto-sandbox mechanism (however not all programs will be virtualized I believe).

That being said, it’s important to develop a clever memory scanner/code emulation system since this is also very beneficial in terms of heuristic analysis (dynamic but not the same as what I’ve been discussing throughout this thread).
--------------------------------------------------

I do apologize for the delay in the posting of this thread by about a week, I was unable to finish it the other day… It’s fairly small compared to my other threads however hopefully it is still found beneficial by someone.


Stay safe and good luck,
Wave. ;)
 
Joined
Mar 13, 2017
Messages
29
OS
Windows 10
Antivirus
ESET
#2
you’ll be able to forget about PatchGuard/Kernel Patch Protection and make use of kernel-mode patching such as MSR hooks on x64 systems, also.
Is this possible to bypass PG/PK without vulnerability?
 

Opcode

Level 28
Content Creator
Joined
Aug 17, 2017
Messages
1,733
#3
Is this possible to bypass PG/PK without vulnerability?
I replied on the other ones I didn't own because you weren't getting a response (I cant even click the profile ?) ...

Yes, you'll need a vulnerability to bypass PatchGuard. DSE and KPP are components of PatchGuard so it is all linked together mate. It only applies for 64-bit environments up from Vista though, thus 32-bit Windows is not affected by it at all.

Windows 10 is much more aggressive with DSE and requires Extended Validation certificates instead of normal, so normal ones whether for kernel mode software signing or not won't work past a very easily Windows 10 patch which everyone should have installed anyway.

DSE is implemented a bit within a module called CI.dll which stands for "Code Integrity", you'll see it is an import of ntoskrnl.exe if you check in Process Hacker. It exports a few routines, which are called at system boot when Windows is starting-up. Depending on flags such as Test Mode being enabled and what not it will work accordingly. One old PatchGuard bypass which should still work to this very day is exploitation of a VirtualBox driver to access the kernel-mode memory where the CI.dll module code is present (it is under ntoskrnl.exe so cannot be accessed from UM without a 0-day (e.g. maybe Physical Memory exploitation) ) to patch the flag value , then load the unsigned driver and then re-patch the flag back to prevent PG from detecting the modification.

It should still work but is a hit n miss situation. It can BSOD if it gets detected before you patch it back but sometimes you can do it quick enough so it isn't caught. Probably checks are done on a timer based scenario so it is unpredictable probably.

You don't even need VBox installed to become a victim of the exploit attack. An attacker can just drop/download it to disk from their launcher and then exploit. The PoC is on GitHub for anyone to view, and malware authors don't care about morals or the law so they don't care about copyright violation...

But on that note.. I doubt you will find any malware in the wild using an exploit like this, even if it is known and documented and not fixed.
 

Opcode

Level 28
Content Creator
Joined
Aug 17, 2017
Messages
1,733
#4
Is this possible to bypass PG/PK without vulnerability?
It isn't ethical to do this anyway, so never do it in commercial software. Only if you are testing for learning. Norton once even told MS they would bypass on purpose but MS retaliated by saying they'd just patch it and leave Norton causing BSOD crashes. Norton didn't like PG being added hahahaha. They didn't do that in the end though of course...

Anyway, an exploit is using a weakness as an advantage. If you bypass PG/KPP or anything related to PG, you're exploiting it = using a vulnerability. Because you'd be using a weakness/something overlooked as an advantage to overcome an obstacle which should be preventing you from doing exactly what you are doing by "bypassing" it. :)