Malware Analysis Trojan.Keylogger (keylogger analysis)

  • This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.


Level 26
Content Creator
Aug 17, 2017


Keyloggers. A keylogger is a type of malware (malicious software) and is very well known for causing a lot of destruction since they became prevalent in the wild; they can be used to spy on someone generally speaking (steal chat logs) or hack into people's accounts through theft of login credentials (even to banking websites which can boil down to theft of money). There are many ways for a keylogger to work, however in this article I'd like to discuss a bit about one of the most common methods used by a keylogger... Hooking of the keyboard.

In Windows, there is a built-in device driver called win32k.sys. This device driver exports many functions for various functionality, however one of them is called NtUserSetWindowsHookEx. This kernel-mode routine is of course undocumented by Microsoft and I doubt that you'll find any real third-party documentation about it online if I am completely honest. In user-mode, there is a Windows module called user32.dll - this module exports a user-mode routine known as SetWindowsHookEx (there is an Ascii version which has the A prefix and a Unicode version which has the W prefix - SetWindowsHookExA or SetWindowsHookExW. The old version would be SetWindowsHookA/W (no Ex* prefix)).

A common method used by keyloggers is to call SetWindowsHookExA/W to set a hook on the keyboard via passing WH_KEYBOARD or WH_KEYBOARD_LL. I believe the "WH" at the start stands for "Windows Hook".

Window hooks will only apply to programs which have user32.dll loaded (typically GUI processes only then). The launcher also must be compiled for the same architecture as the programs being targeted (e.g. a 32-bit process setting a hook on the keyboard will not affect a 64-bit process regardless of user32.dll being loaded in the target process or not and vice-versa). Generally speaking, Windows Hooks can cause issues with performance especially if not implemented properly and depending on the type of hook (e.g. the mouse can be hooked as well, and this can cause invocation to a custom routine setup by the attacker every-time the mouse moves over a program's window which can cause a huge performance decrease!).

NtUserSetWindowsHookEx (WIN32K) should take in 6 parameters: HISTANCE; PUNICODE_STRING (UNICODE_STRING*); DWORD; INT; HOOKPROC; and BOOL. Those should be the data-types for the 6 parameters.

The function NtUserSetWindowsHookEx is Unicode, however there does happen to be an Ascii routine (NtUserSetWindowsHookExAW). The Ascii routine of the function will redirect to NtUserSetWindowsHookEx after ensuring a conversation has been performed so the data can be applied in the Unicode routine (which is responsible for actually performing the operations).

NtUserSetWindowsHookEx should do the following.
- Verify if there are any problems to prevent the continuation.
- Reference an object by its handle (Win32WindowStation (PVOID data-type and present within the EPROCESS structure)).
- Set the hook.

There is a table in win32k.sys which contains handles to all the global hooks set on the system. NtUserSetWindowsHookEx will add to the table of handles when the hook is being set, this allows the system to keep track. This is in kernel-mode memory however this does not mean that you cannot scan for window/mouse hooks or other types of global hooks from user-mode... Whether you believe it or not, it is definitely possible. You'll want to look into gSharedInfo (exported by user32.dll -> you can get the address via GetProcAddress even).

The routine responsible for removing hooks is NtUserUnhookWindowsHookEx. This function will call another routine which will be responsible for actually removing the hook (including removing the data within the structure in win32k.sys which holds the data for the targeted global hook).

There are other routines within win32k.sys for other hook routines. User-mode user32.dll routines such as CallNextHookEx have a kernel-mode alternative as well.

SetWindowsHookExA/W can be used for injection into other processes, it doesn't just have to be used for hooking of windows/keyboard/mouse, etc. You have two options... You can rely on a DLL where a targeted exported routine will be invoked under the context of the target process when the hooked event is triggered (well for all processes which apply for the global hook depending on the circumstances), or you can keep the routine within your own PE.

Malware analysis

We are going to do some quick malware analysis for demonstration purposes. We'll start off with basic static analysis and then we'll try out some API call monitoring. I'll be using a sample I found on GitHub from a repository called theZoo.

After opening up the sample in IDA, I am presented with the WinMain function which had been automatically found for me.

I'm going to check the Imports and see if I find SetWindowsHookExA (USER32) or SetWindowsHookExW (USER32) present.

There is no SetWindowsHookExA/W present.

Below are the Imports.

Address  Ordinal Name                 Library 
-------  ------- ----                 ------- 
00404000         GetModuleHandleA     KERNEL32
00404004         GetTempPathW         KERNEL32
00404008         GetModuleHandleW     KERNEL32
0040400C         GetModuleFileNameW   KERNEL32
00404010         CreateFileW          KERNEL32
00404014         SetFilePointer       KERNEL32
00404018         CloseHandle          KERNEL32
0040401C         GetTempFileNameW     KERNEL32
00404020         FreeLibrary          KERNEL32
00404024         DeleteFileW          KERNEL32
00404028         WriteFile            KERNEL32
0040402C         ReadFile             KERNEL32
00404030         LoadLibraryW         KERNEL32
00404034         GetProcAddress       KERNEL32
00404038         GetStartupInfoA      KERNEL32
00404040         __getmainargs        MSVCRT 
00404044         _initterm            MSVCRT 
00404048         __setusermatherr     MSVCRT 
0040404C         _adjust_fdiv         MSVCRT 
00404050         _acmdln              MSVCRT 
00404054         __p__fmode           MSVCRT 
00404058         __set_app_type       MSVCRT 
0040405C         _except_handler3     MSVCRT 
00404060         _controlfp           MSVCRT 
00404064         _XcptFilter          MSVCRT 
00404068         _exit                MSVCRT 
0040406C         _onexit              MSVCRT 
00404070         __dllonexit          MSVCRT 
00404074         ??1type_info@@UAE@XZ MSVCRT 
00404078         calloc               MSVCRT 
0040407C         exit                 MSVCRT 
00404080         memcpy               MSVCRT 
00404084         memset               MSVCRT 
00404088         _itow                MSVCRT 
0040408C         ??2@YAPAXI@Z         MSVCRT 
00404090         _wcsdup              MSVCRT 
00404094         ??3@YAXPAX@Z         MSVCRT 
00404098         free                 MSVCRT 
0040409C         __p__commode         MSVCRT 
004040A4         MessageBoxW          USER32
This tells us two things can be the case: the sample relies on a dynamic import; or alternatively it will drop another PE which will perform the keylogging.

GetProcAddress (KERNEL32) is statically imported of course, let's check the operand references.

Only one reference, using the CALL instruction. This lets us know that the function does actually get called, but whether the function responsible for calling GetProcAddress will be used during execution is another story. We can check the references of the function responsible for calling GetProcAddress, too.

The function responsible for calling GetProcAddress is called by the WinMain function, which happens to the main entry-point of the Portable Executable (also identified by IDA when the PE was loaded and after the auto-analysis had completed). This let's us know that the chances of this function being invoked at run-time is likely.

We can see a conditional statement being performed, if the value within v4 is not >= 1 (TRUE) then the routine which will call GetProcAddress will be called. The value of v4 is assigned to the return status of another routine.

The return status for that function is based on the return of another function.

That's more like it... We can see file-system operations will be performed. I'll mess with the parameter data for various API calls like CreateFileW so it is more clear.

The sample also imports WriteFile and after checking references to it I know that it can be called at some point. You may notice one of the CreateFileW calls above specifies GENERIC_WRITE for the access right, too... Which indicates it could be used for writing. The TRUNCATE_EXISTING flag means that the file must already exist but the size will be set to 0 -> if the file doesn't exist the call fails. Bear in mind these parameters are not accurate, I am only messing around with enum values. We'll uncover what really happens when we check the API calls dynamically.

There doesn't appear to be much interesting for this static side, I think it is best to just go ahead and start some dynamic analysis to find out what really happens with API Monitoring. The GetProcAddress call was interesting since it attempts to find the address of a function called "sfx_main".

A dynamic import for a function called sfx_main is performed. The program uses a manual type definition for the function via typedef -> used after address is received to call the function in memory with the correct parameters. We are yet to find out which module the function remains in, it certainly isn't linked to the Windows modules itself. My guess is that the sample will drop a DLL to disk which exports a function called sfx_main. I promise I haven't checked this yet, I am writing as I analyse. ;)

The strings are bland too. Time to move onto dynamic analysis, we want to be as quick as possible to find out if the sample is malicious or not. Taking time is good but IMO realistically you will have many samples to get through so unless you're trying to document everything which could take hours for some samples (or a lot) and make a removal guide or full-on analysis, cut to the chase as fast as you can. If you notice anything suspicious, look more closely and spend more time to help prevent missing something important.

I use API Monitor for monitoring API calls usually. In a very rare scenario, I may manually handle it with custom code injection if there is a genuine reason for doing so... Maybe if I wanted to detour NtTerminateProcess (API Monitor never provides the option to break-point on this function) or a function like CreateProcessInternalW. WinDbg would work fine as well though, and a lot easier to do than that since it would be less time-consuming.

I've opened API Monitor and I've set a break-point for the following functions.
1. NtCreateUserProcess (NTDLL) -> break-point before the call is completed.
2. NtResumeThread (NTDLL) -> break-point before the call is completed.
3. NtCreateFile (NTDLL)
4. NtReadFile (NTDLL)
5. NtWriteFile (NTDLL)
6. LdrLoadDll (NTDLL) -> break-point before the call is completed (cautious of many notifications).
7. LdrGetProcedureAddress (NTDLL) -> break-point before the call is completed (cautious of many notifications).
8. SetWindowsHookExA and SetWindowsHookExW (both USER32)
9. CallNextHookEx (USER32)
10. NtAllocateVirtualMemory & NtWriteVirtualMemory (both NTDLL) for the sake of it.

If the sample drops a DLL and attempts to use it, I don't expect anything like manual mapping (also known as "Reflective DLL loading"). Therefore, LdrLoadDll is sufficient.

As soon as I started the sample under monitoring via Static Import (options on API Monitor), I was presented with many API log results. My break-point for LdrLoadDll was hit and it is attempting to load a module called "@1B52.tmp" ("C:\Users\analysis\AppData\Local\Temp\@1B52.tmp"). Interesting!

Look at the value under Name (UNICODE_STRING) -> Buffer (PWSTR). You can see the path of the DLL attempting to be loaded.

You may be wondering, "That is a DLL? It has the *.tmp extension!". You'd be right, it does have an extension for *.tmp, but the file itself is a Portable Executable. It follows the PE File Format, we'll look at this more towards the end.

Since the break-point has been displayed and a fake *.tmp file is attempting to be loaded as a Dynamic Link Library, we should go back to the API logs and see what happened prior to the break-point. Many API calls...

There are actually 483 API calls already but this is my own fault for targeting NTDLL and not all of them will be invoked by a new call. One call to NtCreateFile will cause many to be displayed for the same task. This can be verified by going through parameter data and performing comparisons. I cannot go through all the calls due to how many there are, and it'd be useless because many of them are for the exact same thing (just duplicates of the same operation -> the sample didn't do things again, but because of how Windows works -> calls get triggered on the logs, or maybe it is related to API Monitor software itself).

The sample starts by creating an empty file in AppData\Local\Temp called "@1B52.tmp" via CreateFileW.

At the time of creating the screenshot, my output screen was too small to see it in the image. However, the CreateDisposition parameter to NtCreateFile was set to FILE_CREATE which indicates it is for file creation.

The next call shown in the logs for NtCreateFile targets the newly created file, but this time it requests access for writing to the file using the file handle.

The same file is targeted. It needs to have write access to the newly dropped file so it can write data to it, preferably to store executable code.

The next task is writing data to the dropped file, achieved with the WriteFile (KERNEL32) function, which of course triggers our break-point for NtWriteFile (NTDLL).

The HEX display outputs the data written but it is limited to how much can be displayed, therefore I won't post it here.

The next stage is creating another file under the same directory as the last, but specifying a "3" at the end of the file-name instead of "2". The *.tmp file extension is still used.

NtCreateFile will be hit again but requesting GENERIC_WRITE access rights if the file creation is successful.

A few hundred calls are displayed in the API logs thanks to the break-point on NtReadFile and NtWriteFile. The sample calls ReadFile (KERNEL32) and WriteFile (KERNEL32) to read/write data to the file, and thus our NTDLL break-points get hit as usual. This operation is for storing data within the second created file ("@1B53.tmp"). The first created file is not as affected as much lately, only having being used in only one single write operation.

This is where the LdrLoadDll stage occurs. We'll continue by allowing the call from the break-point window.

This is where sfx_main comes into play, the function is exported by "@1B53.tmp" (actually a DLL). This function will start executing code within the process.

The sample will create a file under the Windows System32 folder ("C:\WINDOWS\system32\28463\DPBJ.001").

Another file will be created under the same folder called "DPBJ.007".

Last but not least, a file with an *.exe extension is dropped to the same folder called "DPBJ.exe".

Since the process is 32-bit compiled and running under WOW64, Windows automatically redirects the file-system operation to the SysWOW64 folder if you're on a 64-bit environment. On 32-bit environments, this folder will not be present because it will not be needed. This can be disabled by calling a Win32 level WOW64 function regarding Fs redirection, but the sample does not handle this.

Due to this, the API logs will continue to reference "C:\\Windows\\System32\\" but in actual fact the operations are redirected to "C:\\Windows\\SysWOW64\\".

The other dozen hundred API calls are regarding writing data to these dropped files.

DPBJ.006 and DPBJ.007 are both PE files as is the *.exe. The first one does not follow the PE File Format.

The next stage is executing DPBJ.exe by spawning it as a process, which causes our NtCreateUserProcess break-point to get hit.

As I mentioned earlier, System32 is still referenced however it isn't present under there. It is under SysWOW64 due to force-redirection regarding WOW64.

By allowing the break-point -> continue call -> NtResumeThread break-point is hit. I enforce this so I can ensure the newly started program is applied to monitoring scope before it gets a chance to execute even one single line of its own PE code.

After the main thread of the new process has been resumed, the launcher will do a few things before terminating.

1. Create a new file called "key.bin" under the same folder the DPBJ.exe was present (SysWow\\...\\...).
2. Create another file called "AKV.exe" under the same folder.
3. Write to both of the files.

key.bin stores the following contents.

              ,RGA3Y3A-M3D88-T3HU5-T28TM-G47A S-SFTD7-624JCKimberley Ronald                        ÿÿÿÿ
AKV.exe stores executable code (its a PE).

The main launcher is now terminated, therefore dynamic analysis entirely focuses on DPBJ.exe which is still running.

The sample will attempt to load DPBJ.006 as a module via LoadLibraryA.

What happens next? A hook request for the keyboard (WH_KEYBOARD) is performed by DPBJ.exe, attempting to use the previously loaded module DPBJ.006 for the call. The way WH_KEYBOARD flag works is you will have a function exported by another module invoked every-time your request (applied on the hook) is triggered. This means there is a function exported by DPBJ.006 which will be called every single time I type a key on the keyboard once the hook has been set.

The sample will set another windows hook using the other dropped DLL, DPBJ.007, for WH_CALLWNDPROC, providing the ability to intercept messages to windows. You can learn more about Windows Hooks here: Hooks Overview (Windows)

The sample will steal from the clipboard and save it to a new file within the folder with HTML formatting.

The sample is also capable of stealing screenshots from the clipboard.

All of this is related to log files for storing data.

The log files which contain HTML formatting can of course be viewed by the local browser.

If you check the exports of DPBJ.006 you will find the following.

We can see the DLLs are important for the sample, and have a big responsibility with the keylogging payload for this sample.


Feel free to experiment with keyloggers while applying analysis techniques, you could even find the same sample I used (on a repository which stores malware samples at GitHub, theZoo) and investigate further than I did. Just remember to stay cautious and ensure you are in a safe environment during testing... The last thing you want to do is end up infecting your Host system or actually have sensitive data stolen.

Remember that there are many different types of keyloggers. The sample demonstrated within this thread uses a known technique, but other techniques are applied by different keylogger samples.

Thanks for reading,
- Opcode.


Dec 23, 2014
Operating System
Windows 10
Installed Antivirus
Many security programs have problems with low-level keyloggers, which work in the kernel. Some AVs like Kaspersky, have some proactive capabilities, like for example monitoring the keyboard device stack to detect keylogger activity.:)
Also in the targetted attacks, one can modify the NtUserGetMessage or NtUserPeekMessage function on the fly. Such keyloggers, if successfully installed, are hardly detected.:(

tim one

Level 20
Jul 31, 2014
Operating System
Windows 10
Installed Antivirus
Brilliant as always @Opcode!!.;)

Yeah like you say there are many ways for a keylogger to work and some of them are also able to use snapshot-logger functions, even if the creation of the snapshot can be quite complex. Under Windows it is possible to take the snapshot of the current screen by calling the function CreateCompatibleBitmap which returns an array of byte ABGR (Alpha, Blue, Green and Red), basically a non-compressed bitmap and then very heavy (several megabytes for a full 1080p display). Sending the raw snapshot to a remote server is possible but really inefficient. Usually keyloggers of this type use a simple solution by converting the bitmap to a PNG file (a few KB) by using the library LodePNG. The library consists of only two files .h and .c: it takes in input a byte array of RGBA values, converting and writing the PNG to disk, and finally returning a further buffer containing PNG data.

Snapshot-loggers can be very dangerous because, even if the passwords are not in clear on the screen (usually replaced by ******), they can heavily violate our privacy and stealing personal data.


Level 26
Content Creator
Aug 17, 2017
Does API Monitor hooks child processes?
You can also attempt to bypass Anti-VM mechanisms with API Monitor. You can break-point before specific functions and fake the parameters to cause an error return = now the sample believes something is not how it is.

Some Anti-VM mechanisms rely on searching for GUIDs such as processes (e.g. VMWare, VirtualBox Guest tools) or registry keys. Therefore you can hook functions like NT equivalents for Win32 API registry functions -> subvert it.

Alternatively you can manually hook NtQuerySystemInformation -> inject your own code before the sample main thread starts (and also do same for all child processes by hooking CreateProcessInternalW). The NtQuerySystemInformation hook will do just what a normal user-mode rootkit will, by hiding specific processes from being found in the returned list. You could then hide VM related processes, sandbox too.

I recon Hybrid Analysis use things like this for its anti-evasion features.