New comprehensive test

Lindrian

New Member
Thread author
Sep 8, 2011
33
Hello

Im thinking about making a new comprehensive security suite test. I was thinking of something along these lines:

1) All these online tests you see are done with random links. This doesnt give a decent comparison to other softwares.
2) They do not very often parse the popups properly
3) They dont try the software on an infected system and test its cleaning capabilities.

So I was thinking of making a test with all the big names like this:
1) Install all the softwares on a clean version of Windows 7 x64 Enterprise
2) Briefly go through each one.
3) Instead of going to each link individually, I will download them prior to testing. I then update all softwares at the same time and then unplug internet (so they all test on the same grounds).
4) I then scan the folder and remove everything that was detected by signatures (as those are not interesting, just waste time for now) and then execute the "real" 0day malware. I then see how the software handles these threats and finally draw a simple conclusions.
5) I then install the products on new fresh system (same update definitions as earlier, unplug from internet) and infect it, then try to clean it with the software.
6) When that is done I draw another conclusion about the entire software performance.

I will probably release this in a PDF too.

What do you guys think?

I am unsure about the "unplugging from internet" thing but I will probably not manage to test all programs in 1 day which means that some might get an edge if they update definitions etc.

What do you think?

I will test the following:
- Kaspersky
- Norton
- Comodo
- Online Armor++
- Vipre
- Bitdefender
- ??
 

McLovin

Level 76
Verified
Honorary Member
Malware Hunter
Apr 17, 2011
9,224
Sounds like a good idea. I would lead it connected to the internet only because you will have people in the world that will have it connected the whole time while not even knowing there was any sort of malware.

You could also try all of the AV software vendors.

Just a question will you be talking in it?
 

jamescv7

Level 85
Verified
Honorary Member
Mar 15, 2011
13,070
That's nice, another twist for testing procedure. Cause mostly I saw any reviews were during the test as internet connectivity was activated.

Just make a double check that its up to date before unplug in the internet. And everything would be zero day samples.
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
Yes, I will be talking in the review.

I have this morning created a program that will snoop malc0de.com and download all samples for a given date. I can now for example download all malware files that are max X days old. I am testing it atm, downloading all files that are max 2 days old. It is working very well! This will ofc help the testing procedure to give more accurate results.

I am however still not sure on how to do it, leave it connected to the internet or not. It's up for you to decide. Convince me either way!
 

McLovin

Level 76
Verified
Honorary Member
Malware Hunter
Apr 17, 2011
9,224
Well leaving it connect would be good to see what it downloads after you removed it. If there is still some on the system. We get to hear that sexy voice of yours :p
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
McLovin said:
Well leaving it connect would be good to see what it downloads after you removed it. If there is still some on the system. We get to hear that sexy voice of yours :p
Yes I'm thinking I will leave it connected. A few days will most likely not be too bad.

I can also say I will install Flash & Java on the system. Is there anything else I should install?

This program I've made is really genius lol. Just now in testing I have downloaded over 250 malware samples dated max 2 days old. Pretty cool. Some are duplicates though, I will create a program that cleans those up too.
 

McLovin

Level 76
Verified
Honorary Member
Malware Hunter
Apr 17, 2011
9,224
Don't think there is any others. Think that you are set now :p
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
Okay. Emsisoft told me to test their internet security suite and NOT OA++.

The list is currently:
- Kaspersky internet security
- Emsisoft internet security
- Norton internet security
- Bitdefender internet security
- Comodo internet security
- Vipre internet security

I think this is enough. I can add max 1 more.
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
I will try f-secure aswell.

I have finished testing emsisoft internet security. Sad to say so far I'm not impressed. It did not do very good at all.

I will before hand say excuse my poor English (havent spoken this much in years) and for the things I might have done wrong. This is just a rough cut so far. I will upload it when everything is done and type up a proper review in pdf.

Edit:
Jesus christ, camstudio JUST crashed on me after a 1 hour review. UGH. I'll see if I can redo this another day. God this is annoying.
 

DiabloBlack

New Member
Nov 5, 2011
193
Sounds like you have a great plan in the making. Running against security suites this go round, maybe test the stand alone AV products next and then all the firewall products in another test. It's a lot of work but sounds like fun.

Would you be willing to share your program for grabbing the malware off malc0de? I've been looking for a way to get a folder filled with malware for testing like I see posted from others but have no clue where they download it all.
 

McLovin

Level 76
Verified
Honorary Member
Malware Hunter
Apr 17, 2011
9,224
Glad you have finished some. We await for the video :p
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
McLovin said:
Glad you have finished some. We await for the video :p
As I said, when I tried to save the video camstudio crashed. Is there any other better program than camstudio?
 

McLovin

Level 76
Verified
Honorary Member
Malware Hunter
Apr 17, 2011
9,224
Lindrian said:
As I said, when I tried to save the video camstudio crashed. Is there any other better program than camstudio?

Yes there is it's called Camstudio. This one is free, but I think Camtasia is the best for desktop screen recording.
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
DiabloBlack said:
Sounds like you have a great plan in the making. Running against security suites this go round, maybe test the stand alone AV products next and then all the firewall products in another test. It's a lot of work but sounds like fun.

Would you be willing to share your program for grabbing the malware off malc0de? I've been looking for a way to get a folder filled with malware for testing like I see posted from others but have no clue where they download it all.

I will share it later, for now I'm still developing it.
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
OK program seems to work fairly fine now.

I am going out of town for a few days, I will have to do the reviews when I am back.

Updated list:

- Kaspersky internet security
- Emsisoft internet security
- Norton internet security
- Bitdefender internet security
- Comodo internet security
- Vipre internet security
- Fsecure internet security
- Eset internet security

total: 8
 

McLovin

Level 76
Verified
Honorary Member
Malware Hunter
Apr 17, 2011
9,224
Lindrian said:
OK program seems to work fairly fine now.

I am going out of town for a few days, I will have to do the reviews when I am back.

Updated list:

- Kaspersky internet security
- Emsisoft internet security
- Norton internet security
- Bitdefender internet security
- Comodo internet security
- Vipre internet security
- Fsecure internet security
- Eset internet security

total: 8

We will be awaiting your arrival :p
 

Lindrian

New Member
Thread author
Sep 8, 2011
33
Okay I am sorry to say but I have too much on my plate for now. The testing is delayed until further notice.
 

About us

  • MalwareTips is a community-driven platform providing the latest information and resources on malware and cyber threats. Our team of experienced professionals and passionate volunteers work to keep the internet safe and secure. We provide accurate, up-to-date information and strive to build a strong and supportive community dedicated to cybersecurity.

User Menu

Follow us

Follow us on Facebook or Twitter to know first about the latest cybersecurity incidents and malware threats.

Top