Suggestions for Malware Vault testers

Discussion in 'Community Feedback' started by TheMalwareMaster, Aug 8, 2017.

  1. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    Good morning... Today, I'd like to make some suggestions to the Malware Vault testers. Even if I have stopped testing there for a long time, there is still something I'd like to say to help the testers.
    In my opinion, everyone should provide a screenshot of the security product version and update (not all testers do this). The update is the most important: we are humans and we may forget to update the signatures. Even if it's really rare to forget that, it should be provided. My second suggestion is about the second opinion scanners usage. In my opinion, if all the samples don't even touch memory and are quarantined by the product, there is no point in doing that. For example, let's say that a tester is using Avira free on a malware pack of 10 items. 4 are detected by local signatures and 6 are blocked on execution by the cloud. At this point, there is no need of a second opinion scan. The same could be said for VoodooShield, COMODO (if set at default-deny, without the sandbox) and avast hardened mode (even if HM doesn't quarantine the sample, that would be the unique left-over) and all products which with a similar mechanism, or that statically detect all samples. Second opinion scanners are more than welcome when a behavioural blocker removes a sample (there may be left-overs). Let me know your thoughts about this
    Regards..
     
  2. brod56

    brod56 Level 11

    Feb 13, 2017
    543
    1,517
    Studant
    Portugal
    Windows 10
    Default-Deny
    I agree. I may start testing some time in the future, and I'll certainly follow your suggestions.
     
    kev216 and SHvFl like this.
  3. silversurfer

    silversurfer Level 39
    Trusted AV Tester

    Aug 17, 2014
    2,894
    31,363
    Germany
    Windows 10
    Microsoft
    #3 silversurfer, Aug 8, 2017
    Last edited: Aug 8, 2017
    Second opinion scanner are able to find remnants of malware by testing dynamically (Execution of samples are missed by signatures), In this case the final system status would be stated as "not clean" because of unactive remnants of malware. Autoruns of payloads/dropper could be detected as well by second opinion scanner. Both examples should be enough to understand why the testers are need to check the systems after the whole testing procedure.
     
  4. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    I agree with the use of second opinion scanners to check the system when the samples are able to enter memory and are (or not) blocked by behavioural blockers. But signatures and cloud (usually) stop the malware execution, so it won't get into memory, and it will be unable to drop any file or modify the system. If all the samples are blocked this way, there is no need of a second opinion scanner
     
    SHvFl likes this.
  5. silversurfer

    silversurfer Level 39
    Trusted AV Tester

    Aug 17, 2014
    2,894
    31,363
    Germany
    Windows 10
    Microsoft
    #5 silversurfer, Aug 8, 2017
    Last edited: Aug 8, 2017
    How do you know if the samples are blocked fast enough before they could be harmed your system ? :rolleyes:

    It seems to me you just want to state your opinion as fact and don't want to understand what I've tried to explain!

    I'm done here... We need an expert on this topic ;)
     
  6. SHvFl

    SHvFl Level 32
    Content Creator Trusted

    Nov 19, 2014
    2,153
    16,391
    Supermodel for McDonald's
    Europe
    Windows 10
    Emsisoft
    Do we even care about signatures though? These days malware morph by the hour if not faster and i wouldn't really care if the few days malware tested has a signature or not.
     
  7. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    #7 TheMalwareMaster, Aug 8, 2017
    Last edited: Aug 8, 2017
    I'll try to explain better. If all the samples are detected statically, I think we will agree that the system is clean without any doubt. Considering the cloud part, or considering default deny products: you are usually running process explorer while executing your samples. If the sample doesn't even appear there, or no new processes are created, the malware couldn't have harmed the system at all. I will make two examples to explain myself better.
    1st case: https://malwaretips.com/threads/signed-malware-downloader.74330/#post-659957

    Here is pretty evident that COMODO blocked the two samples before execution, because they didn't even appear in process explorer, and no new processes were created. So, in my opinion, there is no need in using second opinion scanners here



    2nd case:
    https://malwaretips.com/threads/8-8-17-20.74345/#post-659953
    Here the system was completely clean, until I ran that .LNK file. A new process was created (see dynamic) and then comodo blocked wscript.exe. In this case, running a second opinion scanner is the proper action, because that malware could have performed other malicious actions (but, in the end, the system was clean too)
     
  8. Parsh

    Parsh Level 24
    Trusted AV Tester

    Dec 27, 2016
    1,328
    12,031
    7 Islands of Bombay
    Windows 10
    Default-Deny
    Hey, thanks for suggesting things ;)
    As you know, the product version is clearly mentioned in the directly visible section of the posts, though a concern can be if the product is being tested without being upgraded (that can be known by just looking at the version mentioned in Product name testers mention).
    Regarding sig updates, it will kindof be a muscle memory for the regular testers in case they've not set AV to auto-update. Overall, it sure will be a good thing to do but not everyone provides full screenshots (can't verify update time with system-shown time) and not all AVs show an "updated xx mins ago", leading to a partial loss of intended result. It will be better to provide the time difference between update time and report-posting time.

    Voodooshield and Avast Hardened Mode aren't tested for generality of testing. Regarding Comodo, sure if the sample is blocked immediately before execution (as you say - before reaching memory), the result might be considered as "clean". However, on paper and perhaps practically, there can be exceptions, known and unknown. That's what security policies, attack vectors and fixes are all revolving around.
    Say a malware is carrying a certificate trusted by Comodo's "Trusted Vendor List". It executes without fear and one of the things it does is it forks two new malicious process, one is (trusted) signed process and the other is not signed. Say the unsigned child process has the same name as that of the original sample. The original sample terminates in msec already without you noticing (from what I've read, Windows processes are independent. Children may not be terminated even if the parent is unless made to do so). Say both child processes are set to autorun on reboot and not before. The unsigned one will be contained/sandboxed and the tester might (probably) interpret that the original sample was contained! However the signed trusted process will run after reboot. Your PE may not be opened at that time immediately on reboot (for inspecting) OR you might not be able to know that this malicious process (the trusted signed one) is running unless VT shows a detection score beside it.
    The above was a basic example that may or may not be feasible. Regarding cloud, one cannot be sure that all AV clouds block the main executable process itself unless validated or unless deemed unsafe right? Some AVs might have a partially different policy. Or there can be some slip. All clouds may not ensure a 100% blocking of files before they are assigned memory blocks and begin running.
    As I said, Comodo and anti-executables are a different case. Even then, some of those anti-exes might not monitor all kinds of vectors or execution locations in their default offerings.
    There can be unknown or unexpected reasons causing incomprehensible system changes during testing. Comodo might show that the process is contained via GUI but there was some glitch or some bug (without an error message) failing proper implementation (rare chances though) ..
    Scanning after dynamic testing is done to resolve any kind of possibilities of infection or presence of remnants, rare or not.
     
  9. askalan

    askalan Level 9
    AV Tester

    Jul 27, 2017
    425
    2,748
    Germany
    Linux
    Doctor Web
    The screenshots are very important to show others that the test results are legit. I do screenshots that show updated Zemana on all my tests and everyone should do this too!
     
    frogboy likes this.
  10. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    #10 TheMalwareMaster, Aug 8, 2017
    Last edited: Aug 8, 2017
    I agree with your post but, regarding this part.. If the malware isn't blocked, you will notice it running in ProcessExplorer, right? Then, one should check with second opinion scanners. Which exceptions do you mean?
    An other example I can think of is Avira free. It basically uses only signatures+cloud. Testing it, you will easily notice that the cloud blocks immediatly malware from running, like COMODO. Avira is like: 100% blocked or completely infected, because it has no behavioural blockers (it can detect EXE files spawned by javascript files though, without detecting the downloader)
     
    rockstarrocks and Parsh like this.
  11. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    #11 TheMalwareMaster, Aug 8, 2017
    Last edited: Aug 8, 2017
    https://malwaretips.com/threads/ransomware-double-team-06-08-2017.74294/#post-659330
    Here is an other example in which, in my opinion, second opinion scanners are not needed. Even if I can't understand why to run this kind of test (Why did the tester run only the dynamic test of zemana, when it can be seen that the two samples are already detected statically in the post above? The post above is an other one in which second opinion scanners are not needed, by the way). Both the samples were detected statically by Kaspersky. Then the tester tested zemana. Zemana, in this case, showed the notification used when the samples are detected by signatures. Even if the tester didn't show process explorer at the moment of running these samples,they didn't run in memory at all, because zemana had a signature for them. In this case, the result of the second opinion scanner would be all clean for sure.
     
    askalan likes this.
  12. Parsh

    Parsh Level 24
    Trusted AV Tester

    Dec 27, 2016
    1,328
    12,031
    7 Islands of Bombay
    Windows 10
    Default-Deny
    Okay, to rectify, the example will be modified to say that the child processes try to execute without requiring a reboot.
    I indicated a case where the parent process terminates fast.. and one of its child processes is not signed and carries the same name as the main sample (while the other child process being trusted and signed may be allowed to run). This unsigned child process has the same name as the main sample name and hence the tester might (probably) interpret that the original sample was contained, however actually a child process was contained and another child process could run.
    Non-automated testing can not be perfect in documenting actions.
    I just mean that undocumented and unknown situations can occur in the security paradigm that we might not expect to be valid at the present moment. This is on paper, but can be made practical.
    The above example was a trick, there can be sophisticated modifications of ways of attack to challenge the modeled way of working of the AV technologies.
    Yes. It might hold true for many but not necessarily for all cloud AVs you know. Also if there's some kind of bug or some mis-implementation occurs due to X unexpected technical reason, the normal way of working of an AV component might be hampered in exceptional cases. Practical? Cannot bet.
    Avira might not exactly have a dedicated BB but they say that their so called 'sensor rules' are equivalent to comparing the activity of processes to those known as dangerous. This can be taken as a variation of BB, strong enough or not.
     
  13. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    Yeah, it's Always better to scan with a second opinion scanner.... But, when all samples are detected statically... It's a complete waste of time
    I still can't understand this case well. Is it related to COMODO in particular? (you are talking about containment). Please note that in my COMODO settings I disabled the sandbox and replaced the action with "block" for unrecognised files. This way, none of the files will even go in memory
     
  14. Winter Soldier

    Winter Soldier Level 25

    Feb 13, 2017
    1,466
    10,341
    PLC programmer - Robotics industry
    Wormhole
    Windows 10
    Emsisoft
    I think are not necessary technical justifications.
    Running an on demand scanner at the end of the test is never a waste of time because if this were a problem then we shouldn't do malware testing.
    Please consider that we never completely know the behavior of the sample we go for testing that could have fragmented code designed to act in a non-conventional way when we think it is detected and neutralized by the tested security product.
    In my opinion, it is necessary to perform a final scan (even with more products), as second opinion to get new information, without "if" and without "but".
     
  15. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    I can't really agree with this... If all samples are detected statically, the tester hasn't even tried to run the malicious code
     
  16. Winter Soldier

    Winter Soldier Level 25

    Feb 13, 2017
    1,466
    10,341
    PLC programmer - Robotics industry
    Wormhole
    Windows 10
    Emsisoft
    I don't work in the HUB but if I not mistaken, usually the guys are launching the malware anyway, even if it is statically detected.
    This is to test BB technology regardless of signatures, it is the correct approach and a second scan at the end of the test makes sense imo.
     
  17. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    Scanning is indeed right if the malware is able to run in memory. But, if it's not, that makes no sense
     
  18. Winter Soldier

    Winter Soldier Level 25

    Feb 13, 2017
    1,466
    10,341
    PLC programmer - Robotics industry
    Wormhole
    Windows 10
    Emsisoft
    You should know that many malware run in memory without showing any active processes using Process Explorer, Process Hacker, etc.
    Malcoder can hide active tasks, and do many other things by writing code, this is directly proportional to how complex the code is.
    But if this becomes a loop I leave the word to those who are more competent than me in this specific context.
     
  19. TheMalwareMaster

    TheMalwareMaster Level 19
    Trusted

    Jan 4, 2016
    931
    5,464
    Europe
    Windows 10
    Default-Deny
    I want to stop this discussion because we can't find an agreement.
     
  20. cruelsister

    cruelsister Level 32
    Trusted

    Apr 13, 2013
    2,131
    12,418
    NYC
    Before you close the topic, if I may add one thing about 2nd opinion scanners- there is an unfortunate presumption that running something like MB or HMP after testing the primary product and getting a clean bill of health is proof that the system is not infected.

    This is far from the case! As I have shown previously the 2nd opinion scanners also have their deficiencies and are not 100%. So by running any of these after a test the best that can be said is that MB, HMP, Zemana, also did not find anything. This is a great deal different than stating that the system is actually clean.
     
Loading...
Similar Threads Forum Date
Emsisoft Antimalware Suggestions Emsisoft Jul 4, 2015
Help Me Decide IPS Monitor Suggestions Needed. Compare Hardware Oct 28, 2017
Help Me Decide Need to purchase a new phone. Any suggestions? Compare Hardware Aug 20, 2017