ted114

New Member
Actually, my main question is, how does runtime analysis really works on AVs perspectives. I cant really find an in-depth article online about how do they work, what are the common techniques that they use to approach a specific situation, etc.

So far, in my understanding of this runtime/behavioural analysis of AVs is that, for them to understand and analyze how a process behaves, they have to make API hooks (I hope you understand here, I cant explain much). And this gives them the ability to judge if a spcific program is malicious or not by analysing the APIs it called.

1) How do they know which process called an API that they hooked from kernel-space?

Say for example, an kernel mode driver of an AV placed a hook on an API, Program X->NtQuerySystemInformation->HookedZwQuerySystemInformation->ZwQuerySystemInformation. How would the kernel mode driver be able to identify who called the NtQuerySystemInformation (which in turn, leads to the execution of the hook-function)? Since we know, AVs cant simply rely on user-space API hooks for these scenarios, so they have to do it in kernel space.


2) And how do they know that a running process does some, can-be-considered, malicious things without even calling an API? like a decryption loop perhaps?
 

Andy Ful

Level 40
Content Creator
Trusted
Verified
...
1) How do they know which process called an API that they hooked from kernel-space?

Say for example, an kernel mode driver of an AV placed a hook on an API, Program X->NtQuerySystemInformation->HookedZwQuerySystemInformation->ZwQuerySystemInformation. How would the kernel mode driver be able to identify who called the NtQuerySystemInformation (which in turn, leads to the execution of the hook-function)? Since we know, AVs cant simply rely on user-space API hooks for these scenarios, so they have to do it in kernel space.
...
Because of PatchGuard, actually, most AVs do not rely on kernel-mode hooking, but rather on user-mode hooking.
https://breakdev.org/defeating-antivirus-real-time-protection-from-the-inside/
https://ired.team/offensive-security/bypassing-cylance-and-other-avs-edrs-by-unhooking-windows-apis
https://s7d2.scene7.com/is/content/cylance/prod/cylance-web/en-us/resources/knowledge-center/resource-library/white-papers/Universal_Unhooking.pdf
https://2016.zeronights.ru/wp-content/uploads/2016/12/You’re-Off-the-Hook.pdf

If I correctly remember, sometimes AVs can use hypervisors for kernel-mode hooking.
Furthermore, the mini-filter drivers can be used instead of SSDT hooking (to avoid PatchGuard).
 
Last edited:

ted114

New Member
Because of PatchGuard, actually, most AVs do not rely on kernel-mode hooking, but rather on user-mode hooking.
https://breakdev.org/defeating-antivirus-real-time-protection-from-the-inside/
https://ired.team/offensive-security/bypassing-cylance-and-other-avs-edrs-by-unhooking-windows-apis
https://s7d2.scene7.com/is/content/cylance/prod/cylance-web/en-us/resources/knowledge-center/resource-library/white-papers/Universal_Unhooking.pdf
https://2016.zeronights.ru/wp-content/uploads/2016/12/You’re-Off-the-Hook.pdf

If I correctly remember, sometimes AVs can use hypervisors for kernel-mode hooking.
Furthermore, the mini-filter drivers can be used instead of SSDT hooking (to avoid PatchGuard).
Thank you for that answer.

I actually cant understand something else. So does AVs does it this way?

Program X => Static Analysis => Dynamic Analysis (Emulation, something like AVASTs CyberCapture) => if malicious, stop, else execute on the real environment.

Is that how an AV operates? like, after the emulation, if not malicious, it executes it again on a separate procress that is not emulated anymore? (but ofcourse some functions is still hooked by the AVs to protect itself)

Or does the AV intercepts the real creation of creating the process (using the PsCreateProcessNotifyRoutine) and just run the program under an emulated environment.

So basically my main question is, does a program go thru an emulator (for dynamic analysis) and executed again to the real-unemulated environment? or does it run inside an emulator (emulated-environment) all the time (and not executed again) ?

thank you for your time.
 
  • Like
Reactions: oldschool

Andy Ful

Level 40
Content Creator
Trusted
Verified
If you mean hooking, then AV stops for a moment the execution of real code of the executed program. The hooks are prepared for some popular APIs. Next, AV modules analyze that code, which is placed somewhere in the memory. After this, the code is blocked (if suspicious) or allowed. Here is an example:
"Security software will hook specific user space API functions that are commonly used by malware. For example, a code hook installed on winsock.connect can examine the IP and port of an outgoing network connection and decide whether the connection should be allowed or blocked. A combination of hooks installed on OpenProcess, VirtualAllocEx, WriteProcessMemory, and CreateRemoteThread detect malicious process injection. "

AV can also run the program in the sandbox to analyze the effects of running it, but this is another feature.

The PsCreateProcessNotifyRoutine is a routine in the kernel, it can be installed by a driver. It can be also used by AV to monitor process creation and termination events in the Windows kernel.
 
Last edited:

ted114

New Member
If you mean hooking, then AV stops for a moment the execution of real code of the executed program. The hooks are prepared for some popular APIs. Next, AV modules analyze that code, which is placed somewhere in the memory. After this, the code is blocked (if suspicious) or allowed. Here is an example:
"Security software will hook specific user space API functions that are commonly used by malware. For example, a code hook installed on winsock.connect can examine the IP and port of an outgoing network connection and decide whether the connection should be allowed or blocked. A combination of hooks installed on OpenProcess, VirtualAllocEx, WriteProcessMemory, and CreateRemoteThread detect malicious process injection. "

AV can also run the program in the sandbox to analyze the effects of running it, but this is another feature.

The PsCreateProcessNotifyRoutine is a routine in the kernel, it can be installed by a driver. It can be also used by AV to monitor process creation and termination events in the Windows kernel.
I actually meant CPU Emulation.

But about hooking, do you think kernel-modules that placed hooks on these APIs has the ability to know which specific user-mode program called that API?
 
  • Like
Reactions: oldschool

Andy Ful

Level 40
Content Creator
Trusted
Verified
I actually meant CPU Emulation.

But about hooking, do you think kernel-modules that placed hooks on these APIs has the ability to know which specific user-mode program called that API?
You do not need the kernel modules that placed hooks on these APIs, to find out which specific user mode process called a specific API. You can simply use a combination of hooks from my previous post. The API function OpenProcess will give you the open handles of the processes. Next, you can take advantage of other hooked APIs to get additional information if any process is suspicious. Furthermore, If you stopped the execution flow of the process, then you can use your own modules to analyze the code (already executed and not-yet-executed) and changes made in the system.
 
Last edited:

ted114

New Member
You do not need the kernel modules that placed hooks on these APIs, to find out which specific user mode process called a specific API. You can simply use a combination of hooks from my previous post. The API function OpenProcess will give you the open handles of the processes. Next, you can take advantage of other hooked APIs to get additional information if any process is suspicious. Furthermore, If you stopped the execution flow of the process, then you can use your own modules to analyze the code (already executed and not-yet-executed) and changes made in the system.
"You do not need kernel-modules" you mean, AVs(in general, if not all) just use user-mode (dll-injection) + hooking (to all processes) to know and analyze what APIs (and thus, analyze how the program interacts with the system) ??

I see, so AVs just use kernel modules to protect itself from being attacked by malwares or send some notification to some of it's user mode processes (like, when a process is created).

Thanks.
 
  • Like
Reactions: Andy Ful

Andy Ful

Level 40
Content Creator
Trusted
Verified
"You do not need kernel-modules" you mean, AVs(in general, if not all) just use user-mode (dll-injection) + hooking (to all processes) to know and analyze what APIs (and thus, analyze how the program interacts with the system) ??

I see, so AVs just use kernel modules to protect itself from being attacked by malwares or send some notification to some of it's user mode processes (like, when a process is created).

Thanks.
The malc0ders usually use APIs in userland, so AVs hunt for those APIs to fight the malware.
 

Andy Ful

Level 40
Content Creator
Trusted
Verified
I believe he means AV devs will look for the same APIs to use in their own way as a countermeasure.
The way of using some APIs by malware usually differs from using them by most clean applications. So, AVs can hook on the APIs commonly used by malware and look for suspicious patterns. The legal applications may also use the hooked APIs, so the AV developer has to find the patterns which will be characteristic to malware files.
 
Last edited: