We can create a set of rules which will identify a particular malicious behaviour (in a static analysis) without actually having to emulate the execution processes (as in behavioural detection).
For example, we can scan the import table to look for suspicious imports, eg: WriteProcessMemory or NtWriteVirtualMemory ~ If we detect either of these imports, we assign a score to the file. For example WriteProcessMemory will have a score of 0.5 (in our example) and NtWriteVirtualMemory being a less commonly used API (and deeper within the OS) will have a higher score of 1 (as it is more often abused by malware writers).
We can assign further scores based even on meta data, for example, if the company name is "Microsoft" we assign a score of '1' (relying on our whitelist or a hash check to exclude this file if it's official, otherwise we may have a false positive, but this way we're less likely to have a false negative) or if there is no company name we assign a score of 0.5
.. We can also look for suspicious strings in the file, for example any web address that ends in .exe/.dll etc will immediately get a score of 1. A malformed EXE header (eg: non standard MZ stub, wrong size information etc) gives us a score of 1.
At the end of all the checks we get a total 'risk score' which in this case let's say is 4.5. Our maximum 'safe' score is 3, so we have two choices. Either we can mark the file as suspicious eg: !suspicious or we can queue the file for further checks, eg: emulation or virustotal scanning.
You can actually combine more specific rules to detect individual virus families. For example, if the file creates a process with a random five letter name and also drops a similarly named dll in C:\Windows\System32 we can use regular expressions to match both of these rules in a file (byte matching) and then we get a more specific detection than just 'suspicious'.
A simple example, the following sequence of bytes will detect some forms of UPX packer. The question marks (??) represent wildcard bytes
Code:
60 BE ?? ?? ?? ?? 8D BE ?? ?? ?? ?? 57 83 CD FF EB 10 90 90 90 90 90 90 8A 06 46 88 07 47 01 DB 75 07 8B 1E 83 EE FC 11 DB 72 ED B8 01 00 00 00 01 DB 75 07 8B 1E 83 EE FC 11 DB 11 C0 01 DB
^ Interestingly enough, the beginning of this byte sequence will also match files packed with ASPack, and so on
Hope that helps