Suggestions for Malware Vault testers

[TheMalwareMaster]

Good morning... Today, I'd like to make some suggestions to the Malware Vault testers. Even if I have stopped testing there for a long time, there is still something I'd like to say to help the testers.
In my opinion, everyone should provide a screenshot of the security product version and update (not all testers do this). The update is the most important: we are humans and we may forget to update the signatures. Even if it's really rare to forget that, it should be provided. My second suggestion is about the second opinion scanners usage. In my opinion, if all the samples don't even touch memory and are quarantined by the product, there is no point in doing that. For example, let's say that a tester is using Avira free on a malware pack of 10 items. 4 are detected by local signatures and 6 are blocked on execution by the cloud. At this point, there is no need of a second opinion scan. The same could be said for VoodooShield, COMODO (if set at default-deny, without the sandbox) and avast hardened mode (even if HM doesn't quarantine the sample, that would be the unique left-over) and all products which with a similar mechanism, or that statically detect all samples. Second opinion scanners are more than welcome when a behavioural blocker removes a sample (there may be left-overs). Let me know your thoughts about this
Regards..

 

[brod56]

I agree. I may start testing some time in the future, and I'll certainly follow your suggestions.

 

[silversurfer]

My second suggestion is about the second opinion scanners usage. In my opinion, if all the samples don't even touch memory and are quarantined by the product, there is no point in doing that. For example, let's say that a tester is using Avira free on a malware pack of 10 items. 4 are detected by local signatures and 6 are blocked on execution by the cloud. At this point, there is no need of a second opinion scan. The same could be said for VoodooShield, COMODO (if set at default-deny, without the sandbox) and avast hardened mode (even if HM doesn't quarantine the sample, that would be the unique left-over) and all products which with a similar mechanism, or that statically detect all samples. Second opinion scanners are more than welcome when a behavioural blocker removes a sample (there may be left-overs). Let me know your thoughts about this
Regards..

Second opinion scanner are able to find remnants of malware by testing dynamically (Execution of samples are missed by signatures), In this case the final system status would be stated as "not clean" because of unactive remnants of malware. Autoruns of payloads/dropper could be detected as well by second opinion scanner. Both examples should be enough to understand why the testers are need to check the systems after the whole testing procedure.

 

[TheMalwareMaster]

Second opinion scanner are able to find remnants of malware by testing dynamically (Execution of samples are missed by signatures), In this case the final system status would be stated as "not clean" because of unactive remnants of malware. Autoruns of payloads/dropper could be detected as well by second opinion scanner. Both examples should be enough to understand why the testers are need to check the systems after the whole testing procedure.

I agree with the use of second opinion scanners to check the system when the samples are able to enter memory and are (or not) blocked by behavioural blockers. But signatures and cloud (usually) stop the malware execution, so it won't get into memory, and it will be unable to drop any file or modify the system. If all the samples are blocked this way, there is no need of a second opinion scanner

 

[silversurfer]

I agree with the use of second opinion scanners to check the system when the samples are able to enter memory and are (or not) blocked by behavioural blockers. But signatures and cloud (usually) stop the malware execution, so it won't get into memory, and it will be unable to drop any file or modify the system. If all the samples are blocked this way, there is no need of a second opinion scanner

How do you know if the samples are blocked fast enough before they could be harmed your system ?

It seems to me you just want to state your opinion as fact and don't want to understand what I've tried to explain!

I'm done here... We need an expert on this topic

 

[SHvFl]

Do we even care about signatures though? These days malware morph by the hour if not faster and i wouldn't really care if the few days malware tested has a signature or not.

 

[TheMalwareMaster]

How do you know if the samples are blocked fast enough before they could be harmed your system ?

It seems to me you just want to state your opinion as fact and don't want to understand what I've tried to explain!

I'm done here... We need an expert on this topic

I'll try to explain better. If all the samples are detected statically, I think we will agree that the system is clean without any doubt. Considering the cloud part, or considering default deny products: you are usually running process explorer while executing your samples. If the sample doesn't even appear there, or no new processes are created, the malware couldn't have harmed the system at all. I will make two examples to explain myself better.
1st case: https://malwaretips.com/threads/signed-malware-downloader.74330/#post-659957

Here is pretty evident that COMODO blocked the two samples before execution, because they didn't even appear in process explorer, and no new processes were created. So, in my opinion, there is no need in using second opinion scanners here



2nd case:
https://malwaretips.com/threads/8-8-17-20.74345/#post-659953
Here the system was completely clean, until I ran that .LNK file. A new process was created (see dynamic) and then comodo blocked wscript.exe. In this case, running a second opinion scanner is the proper action, because that malware could have performed other malicious actions (but, in the end, the system was clean too)

 

[Parsh]

In my opinion, everyone should provide a screenshot of the security product version and update (not all testers do this). The update is the most important: we are humans and we may forget to update the signatures.

Hey, thanks for suggesting things
As you know, the product version is clearly mentioned in the directly visible section of the posts, though a concern can be if the product is being tested without being upgraded (that can be known by just looking at the version mentioned in Product name testers mention).
Regarding sig updates, it will kindof be a muscle memory for the regular testers in case they've not set AV to auto-update. Overall, it sure will be a good thing to do but not everyone provides full screenshots (can't verify update time with system-shown time) and not all AVs show an "updated xx mins ago", leading to a partial loss of intended result. It will be better to provide the time difference between update time and report-posting time.

In my opinion, if all the samples don't even touch memory and are quarantined by the product, there is no point in doing that. For example, let's say that a tester is using Avira free on a malware pack of 10 items. 4 are detected by local signatures and 6 are blocked on execution by the cloud. At this point, there is no need of a second opinion scan. The same could be said for VoodooShield, COMODO (if set at default-deny, without the sandbox) and avast hardened mode (even if HM doesn't quarantine the sample, that would be the unique left-over) and all products which with a similar mechanism, or that statically detect all samples. Second opinion scanners are more than welcome when a behavioural blocker removes a sample (there may be left-overs).

But signatures and cloud (usually) stop the malware execution, so it won't get into memory, and it will be unable to drop any file or modify the system. If all the samples are blocked this way, there is no need of a second opinion scanner

Voodooshield and Avast Hardened Mode aren't tested for generality of testing. Regarding Comodo, sure if the sample is blocked immediately before execution (as you say - before reaching memory), the result might be considered as "clean". However, on paper and perhaps practically, there can be exceptions, known and unknown. That's what security policies, attack vectors and fixes are all revolving around.
Say a malware is carrying a certificate trusted by Comodo's "Trusted Vendor List". It executes without fear and one of the things it does is it forks two new malicious process, one is (trusted) signed process and the other is not signed. Say the unsigned child process has the same name as that of the original sample. The original sample terminates in msec already without you noticing (from what I've read, Windows processes are independent. Children may not be terminated even if the parent is unless made to do so). Say both child processes are set to autorun on reboot and not before. The unsigned one will be contained/sandboxed and the tester might (probably) interpret that the original sample was contained! However the signed trusted process will run after reboot. Your PE may not be opened at that time immediately on reboot (for inspecting) OR you might not be able to know that this malicious process (the trusted signed one) is running unless VT shows a detection score beside it.
The above was a basic example that may or may not be feasible. Regarding cloud, one cannot be sure that all AV clouds block the main executable process itself unless validated or unless deemed unsafe right? Some AVs might have a partially different policy. Or there can be some slip. All clouds may not ensure a 100% blocking of files before they are assigned memory blocks and begin running.
As I said, Comodo and anti-executables are a different case. Even then, some of those anti-exes might not monitor all kinds of vectors or execution locations in their default offerings.
There can be unknown or unexpected reasons causing incomprehensible system changes during testing. Comodo might show that the process is contained via GUI but there was some glitch or some bug (without an error message) failing proper implementation (rare chances though) ..
Scanning after dynamic testing is done to resolve any kind of possibilities of infection or presence of remnants, rare or not.

 

[AlanOstaszewski]

The screenshots are very important to show others that the test results are legit. I do screenshots that show updated Zemana on all my tests and everyone should do this too!

 

[TheMalwareMaster]

Regarding Comodo, sure if the sample is blocked immediately before execution (as you say - before reaching memory), the result might be considered as "clean". However, on paper and perhaps practically, there can be exceptions, known and unknown. That's what security policies, attack vectors and fixes are all revolving around.

I agree with your post but, regarding this part.. If the malware isn't blocked, you will notice it running in ProcessExplorer, right? Then, one should check with second opinion scanners. Which exceptions do you mean?
An other example I can think of is Avira free. It basically uses only signatures+cloud. Testing it, you will easily notice that the cloud blocks immediatly malware from running, like COMODO. Avira is like: 100% blocked or completely infected, because it has no behavioural blockers (it can detect EXE files spawned by javascript files though, without detecting the downloader)

 

[TheMalwareMaster]

https://malwaretips.com/threads/ransomware-double-team-06-08-2017.74294/#post-659330
Here is an other example in which, in my opinion, second opinion scanners are not needed. Even if I can't understand why to run this kind of test (Why did the tester run only the dynamic test of zemana, when it can be seen that the two samples are already detected statically in the post above? The post above is an other one in which second opinion scanners are not needed, by the way). Both the samples were detected statically by Kaspersky. Then the tester tested zemana. Zemana, in this case, showed the notification used when the samples are detected by signatures. Even if the tester didn't show process explorer at the moment of running these samples,they didn't run in memory at all, because zemana had a signature for them. In this case, the result of the second opinion scanner would be all clean for sure.

 

[Parsh]

If the malware isn't blocked, you will notice it running in ProcessExplorer, right? Then, one should check with second opinion scanners.

Okay, to rectify, the example will be modified to say that the child processes try to execute without requiring a reboot.
I indicated a case where the parent process terminates fast.. and one of its child processes is not signed and carries the same name as the main sample (while the other child process being trusted and signed may be allowed to run). This unsigned child process has the same name as the main sample name and hence the tester might (probably) interpret that the original sample was contained, however actually a child process was contained and another child process could run.
Non-automated testing can not be perfect in documenting actions.

Which exceptions do you mean?

I just mean that undocumented and unknown situations can occur in the security paradigm that we might not expect to be valid at the present moment. This is on paper, but can be made practical.
The above example was a trick, there can be sophisticated modifications of ways of attack to challenge the modeled way of working of the AV technologies.

An other example I can think of is Avira free. It basically uses only signatures+cloud. Testing it, you will easily notice that the cloud blocks immediatly malware from running, like COMODO. Avira is like: 100% blocked or completely infected, because it has no behavioural blockers (it can detect EXE files spawned by javascript files though, without detecting the downloader)

Yes. It might hold true for many but not necessarily for all cloud AVs you know. Also if there's some kind of bug or some mis-implementation occurs due to X unexpected technical reason, the normal way of working of an AV component might be hampered in exceptional cases. Practical? Cannot bet.
Avira might not exactly have a dedicated BB but they say that their so called 'sensor rules' are equivalent to comparing the activity of processes to those known as dangerous. This can be taken as a variation of BB, strong enough or not.

 

[TheMalwareMaster]

Yeah, it's Always better to scan with a second opinion scanner.... But, when all samples are detected statically... It's a complete waste of time

Okay, to rectify, the example will be modified to say that the child processes try to execute without requiring a reboot.
I indicated a case where the parent process terminates fast.. and one of its child processes is not signed and carries the same name as the main sample (while the other child process being trusted and signed may be allowed to run). This unsigned child process has the same name as the main sample name and hence the tester might (probably) interpret that the original sample was contained, however actually a child process was contained and another child process could run.
Non-automated testing can not be perfect in documenting actions.

I still can't understand this case well. Is it related to COMODO in particular? (you are talking about containment). Please note that in my COMODO settings I disabled the sandbox and replaced the action with "block" for unrecognised files. This way, none of the files will even go in memory

 

[Winter Soldier]

I think are not necessary technical justifications.
Running an on demand scanner at the end of the test is never a waste of time because if this were a problem then we shouldn't do malware testing.
Please consider that we never completely know the behavior of the sample we go for testing that could have fragmented code designed to act in a non-conventional way when we think it is detected and neutralized by the tested security product.
In my opinion, it is necessary to perform a final scan (even with more products), as second opinion to get new information, without "if" and without "but".

 

[TheMalwareMaster]

I can't really agree with this... If all samples are detected statically, the tester hasn't even tried to run the malicious code

 

[Winter Soldier]

I can't really agree with this... If all samples are detected statically, the tester hasn't even tried to run the malicious code

I don't work in the HUB but if I not mistaken, usually the guys are launching the malware anyway, even if it is statically detected.
This is to test BB technology regardless of signatures, it is the correct approach and a second scan at the end of the test makes sense imo.

 

[TheMalwareMaster]

I don't work in the HUB but if I not mistaken, usually the guys are launching the malware anyway, even if it is statically detected.
This is to test BB technology regardless of signatures, it is the correct approach and a second scan at the end of the test makes sense imo.

Scanning is indeed right if the malware is able to run in memory. But, if it's not, that makes no sense

 

[Winter Soldier]

Scanning is indeed right if the malware is able to run in memory. But, if it's not, that makes no sense

You should know that many malware run in memory without showing any active processes using Process Explorer, Process Hacker, etc.
Malcoder can hide active tasks, and do many other things by writing code, this is directly proportional to how complex the code is.
But if this becomes a loop I leave the word to those who are more competent than me in this specific context.

 

[TheMalwareMaster]

I want to stop this discussion because we can't find an agreement.

 

[cruelsister]

Before you close the topic, if I may add one thing about 2nd opinion scanners- there is an unfortunate presumption that running something like MB or HMP after testing the primary product and getting a clean bill of health is proof that the system is not infected.

This is far from the case! As I have shown previously the 2nd opinion scanners also have their deficiencies and are not 100%. So by running any of these after a test the best that can be said is that MB, HMP, Zemana, also did not find anything. This is a great deal different than stating that the system is actually clean.