MRG Effitas Real World Exploit Prevention Test March 2015

[Azure]

Project details: MRG Effitas Real World Exploit Prevention Test March 2015

• Diverse set of exploit kits (12)
• Diverse set of vulnerabilities (16 different CVEs) in the product comparison
• Internet Explorer, Firefox and Chrome exploits used
• Large number of internet security suites and anti-exploit tools – 13 products
• Use of in-the-wild in-memory malware
• Test with an artificial zero-day attack
• Manual test and result analysis
• Combined in-the-wild and Metasploit test
• Sponsored by SurfRight – HitmanPro Alertv3

Source: https://www.mrg-effitas.com/mrg-effitas-real-world-exploit-prevention-test-march-2015/

 

[Azure]

Found these info at Wilders. Will post in case anyone here would like to read the discussion there
http://www.wilderssecurity.com/thre...5-sponsored-by-surfright.374988/#post-2478791

 

[Piteko21]

I already told you that I love hitmanpro?


let's go back to using hitman pro in the browser ...

 

[hjlbx]

The Lohman brothers really improved HMPA.

Lots of really hard work.

It shows.

Better than v. 2.

 

[kiric96]

i wonder how they "do" these test... while HMPA may be a good product i dont really think that they did an equal test for all of the products, neither they use a fair environment

 

[Enju]

http://support.emsisoft.com/topic/1...heidenen-ergebnis-in-diesem-test/#entry129257 (It's in German)
Emsisoft themselves are uncertain why they were even tested since they don't provide anti-exploit. Allowing the exploit to run, but blocking the payload via signatures or behaviour blocker was still counted as failure.

 

[FleischmannTV]

Funny how existing customers, who would have sucked in this test big time, are conveniently spared from being tested, whereas former customers like Emsisoft, who don't even advertise exploit protection, are bludgeoned in this comparative and are not even permitted to utilize their modules properly, so they are much more likely to fail.

 

[Enju]

Funny how existing customers, who would have sucked in this test big time, are conveniently spared from being tested

You are talking about Webroot, am I right? I was wondering why they weren't being tested while Emsisoft was.

 

[hjlbx]

I read the test methodology.

Unfortunately, MRG doesn't specifically state in no uncertain terms - that they only tested exploit protection within the browser.

What they did is there in print. One just has to carefully read what they counted as a block and a fail. Only then is it clear.

That's why they ignored payload blocks by, for example, Emsisoft's Behavior Blocker.

Lots of folks are screaming "foul" over this test, but to be perfectly honest, ignoring a payload block makes sense to me.

In any case, once in a while MRG tests get that "dodgy" feel to them.

This is one of them.

There's always some kind of drama connected to HMPA... either on the forums or published tests.

If given the choice... would you use HMPA 3 or the old version 2 ?

Hah... gotcha.

* * * * *

I use Emsisoft.

Emsisoft openly states that their products provide no anti-exploit protections for compatibility reasons (go look on Emsi forum).

So I use anti-exploit for the browser only - MBAE free.

I do not use any Adobe products, Oracle's Java\JRE, Windows Media Player, Microsoft Office, etc - so as to reduce the attack surface.

 

[randj89]

Instead of naming it real world exploit prevention test, they should name it Hitman Pro Alert test because that's the only one you can see. They put charts of the other products but there's no actual tests shown.... So that makes me wonder how did they get the results..... No proof of concept or anything for me....

 

[hjlbx]

Instead of naming it real world exploit prevention test, they should name it Hitman Pro Alert test because that's the only one you can see. They put charts of the other products but there's no actual tests shown.... So that makes me wonder how did they get the results..... No proof of concept or anything for me....

It's a test pretending to be a test.

It was cobbled together.

Initially, only HMPA 3 was tested.

Surfright requested other security solutions be added for comparison.

Still, HMPA 3 has a lot of improvements over old version 2.

 

[Alexstrasza]

This test is just ...weird.

They even tested products that openly do not possess anti-exploit modules (as a lot of people has stated above... Emsisoft would be a prime example). What is the point of that?

You cannot test a fish by telling it to climb a tree.

If this kind of tests want to be called *fair*, then it should be between HMP.A, MBAE, EMET and Crystal AE.

 

[randj89]

It's a test pretending to be a test.

It was cobbled together.

Initially, only HMPA 3 was tested.

Surfright requested other security solutions be added for comparison.

Still, HMPA 3 has a lot of improvements over old version 2.

I agree!

 

[Enju]

What they did is there in print. One just has to carefully read what they counted as a block and a fail. Only then is it clear.
That's why they ignored payload blocks by, for example, Emsisoft's Behavior Blocker.
Lots of folks are screaming "foul" over this test, but to be perfectly honest, ignoring a payload block makes sense to me.

I just translated what Emisoft said themselves, but why would ignoring a payload block make sense? As long as the exploit has no real system impact it doesn't really matter if it was blocked before or after execution imo.

 

[hjlbx]

MRG only consider it an anti-exploit if it was blocked at the browser level.

Once the payload has been completely downloaded it has left the browser and entered the Windows file system.

 

[bitbizket]

The fun part almost all AV + MAE used as a companion scored 98-100% even Emsisoft.
Seems like a one sided test which does not considered other layered of protection involved.

 

[rienna]

Nice to see. I'm curious if anyones ran any test with MAE free with all the protections turned on. (by default only a fraction are on).
In the free version the only thing you can't turn protection on for in advanced settings is VB scripting protection for anything other than Internet Explorer. Surprisingly enough.

I've personally opted to crank everything up (what could it hurt?) and haven't noticed any negative side effects.