Cylance - Targeted and Bypassed

[Burrito]

Researchers Easily Trick Cylance's AI-Based Antivirus Into Thinking Malware Is 'Goodware'

By taking strings from an online gaming program and appending them to malicious files, researchers were able to trick Cylance’s AI-based antivirus engine into thinking programs like WannaCry and other malware are benign.

www.vice.com

Cylance developed a great machine-learning (ML) algorithm. It's proven to be very effective against malware, particularly 0-day malware.

But... I've said multiple times... once Cylance gets big enough that malware developers start targeting it specifically -- then it's like all other AVs. Malware developers can just keep on tweaking until they figure out how to bypass. Although, it is harder to 'tweak to bypass' ML software. You could tweak thousands of times and get nowhere, whereas with signature based products, there are more efficient ways to pack or obfuscate code until you can make it work.

Cylance is big enough now. Cylance has grown super rapidly in market share. They may pass Kaspersky in market size soon.

I've also stated that Cylance should be run as a 'module' -- as part of a larger defense strategy.

There is a benefit to 'security by obscurity.' If your product is not targeted by malware developers --- obviously that is to your advantage.

It's still a great product -- as part of a layered approach.

And the Cylance guys need to dive into the code and get this fixed.

 

[davisd]

Permeh, who is also the architect of Cylance’s machine-learning engine, said they do use signatures and hard-coded heuristics in their product as well and don’t entirely rely on the machine-learning, but the AI does take precedence in detection.

Now, don't act like it wasn't foretold before.. it was coming sooner or later in light, its just for the those who ignored this and was riding in a AI hype train for a long. To be honest, this reverse-enginering trick which has been discovered and made public, in my opinion, is the greatest thing what has happened with Cylance ever since its been out, because now these news will greatly shake up Cylance and its AI might actually become better and useful one day, once devs find the right balance in the AI algorythms to make it better differ between good and bad files, and fix the high false positives rates which makes gamers pull their hairs out, as currently there is no easy way to exclude detections on endpoints other than loging in via dashboard and managing there what's been detected/whitelist what's needed. They have not reached a point for "AI" to completely replace traditional AVs modules for detecting malware, its just marketing what they are good at pushing towards, to make masses believe. However I do still believe, Cylance is an fine option in a layered approach.

 

[Nightwalker]

Ouch, very interesting read, @Burrito was right all the way about Cylance future.

I am using Cylance Smart Antivirus in my secondary machine in a combo setup and it is running fine, but I am seeing more and more threats that arent detected by it, at least the false positives had lowered too.

 

[Burrito]

To be honest, this reverse-engineering trick which has been discovered and made public, in my opinion, is the greatest thing what has happened with Cylance ever since its been out, because now this news will greatly shake up Cylance and its AI might actually become better and useful one day, once devs find the right balance in the AI algorithms to make it better differ between good and bad files, and fix the high false positives rates...

Here here... that would be an optimal outcome. And... Cylance was quick out of the gate and developed the best ML model. But 'industry talk' is that Cylance may not even be in the top 5 for ML algorithms now. They need to pick it up and get it going again.

Ouch, very interesting read, @Burrito was right all the way about Cylance future.

Laughing... thanks for the credit friend..

 

[ForgottenSeer 58943]

I dropped Cylance the day Blackberry purchased them. I still think it could have a place in a layered setup, it has some benefits for that as Burrito indicates. But honestly, I haven't used it since the Blackberry thing and probably won't.

 

[[correlate]]

We have to be a good Cylance protection program and have moved to a whole new world that differs from previous anti-malware solutions.

Professional people can target any program and may overcome it, but that does not mean it's a bad program
- I chose Cylance because it does not do my own annoying pop-ups and no annoying updates periodically, nor does it consume computer resources

 

[DeepWeb]

Wow! How did Cylance become that popular??

 

[ForgottenSeer 58943]

I checked in with some folks.. This is worse than it appears. There will be significant development time and re-training to overcome this. It literally will require re-engineering aspects of the program. It's been effectively pwned.

Personally, I wouldn't run it at all right now. Not even in combination with something else. Pwned is pwned.

Some really really big, impressive organizations almost exclusively rely on Cylance. One comes to mind that has 30,000+ endpoints with it. They're in trouble.

 

[Burrito]

Wow! How did Cylance become that popular??

Seems like I read about them partnering with somebody new every few weeks.... they have done well with the partnering model.

Genetec Partners With Cylance To Provide AI-Based Antivirus Protection To Its Customers

Genetec Inc., a technology provider of unified security, public safety, operations, and business intelligence solutions, announces that it is partnering with Cylance, a business unit of BlackBerry,...

www.securityinformed.com

It's been effectively pwned.

Personally, I wouldn't run it at all right now. Not even in combination with something else. Pwned is pwned.

Nah.... that's a dramatic overreaction. Seems like you've never been to a blackhat conference. Anything can be pwned if it is directly targeted. At blackhat conferences, there are demonstrations of how to pwn virtually every top capability.

Signature-based products can be pwned easily if directly targeted by standard hacker packing and obfuscation tools. It's not newsworthy.

But... it's harder to pwn good ML code. And that's why it was in the news.

Cylance and the whole ML concept just got brought down a rung... and rightly so. The hype has gone beyond the actual capability.

But good ML code is still great as part of a layered approach.

 

[oldschool]

Poached from Wilders: Skylight Cyber | Cylance, I Kill You!

 

[davisd]

BlackBerry Cylance Guidance on Bypass Disclosure

BlackBerry® Cylance® is aware that a bypass has been publicly disclosed by security researchers. Our research and development teams have identified a solution and will release a hotfix automatically to all customers.

threatvector.cylance.com

BlackBerry® Cylance® is aware that a bypass has been publicly disclosed by security researchers. We have verified there is an issue with CylancePROTECT® which can be leveraged to bypass the anti-malware component of the product.

Our research and development teams have identified a solution and will release a hotfix automatically to all customers running current versions in the next few days.

More information will be provided as soon as it is available.

Hotfix in few days.. I wanna see that, will Cylance once again chimp out and make it look alike it's nothing serious to worry about? As a home user I don't give a flying f, i'd just uninstall it, but if I were running a enterprise with a thousands of endpoints equiped with Cylance..

 

[Burrito]

Poached from Wilders: Skylight Cyber | Cylance, I Kill You!

Good article.

Delineates the really good aspects of ML, and the basis to attack them.

A few quotes from the article--

Once every few years, the cybersecurity world is blessed with the birth of a baby silver bullet. It starts small, with a few enthusiastic parent companies hailing the newborn prince as our savior, telling the stories of its invincible power. A few years and millions of marketing dollars later, it grows and becomes an absolute powerhouse.
...
Such is the story of many “silver bullets” we have seen over the years, and inevitably such will be the story of AI and machine learning.
AI has been touted as the silver bullet to end them all with significant marketing force — after all, if we can teach a machine to think like a human analyst, only with the addition of big data and almost infinite processing power, then surely, we will be able to create an invincible mechanism. A brain so powerful that it could not be fooled by any other mechanism.

Right…

Yep. Anybody who's been around network security for a while has seen the new bright shiny objects come and go..

With a fair bit of knowledge of how the model works in hand, we hypothesized as to how we can actually circumvent and confuse the engine.
Our first hypothesis was to try to make a malicious PE look like one of the files in the whitelist. That is, force the relevant features into the right distance from a whitelisted centroid. We quickly realized that this technique has little chance to work as this mechanism relies on thousands of features, some of which are extremely hard to modify.

And this is the power or ML/AI. It can be difficult to bypass.

Impact and Final Thoughts

We are always amused to see the shock on people’s faces when you tell them that the new security toy they spent millions of dollars buying and integrating can be bypassed. The same goes for new silver bullets, like AI-based security. We are anything but surprised with the results, and we are confident that the same type of process can be applied to other pure AI vendors to achieve similar results.
Why?
Vendors too often approach the security problem with a one-punch solution. Hackers are not wooden dummies, they fight back, and you have to be ready for the counter-punch, constantly innovating and increasing the cost of attack.

Yep. If you specifically target a capability, you will eventually find a weakness. That's just the nature of the beast.

Granted, it is harder to find a bias in an AI model than to bypass a simple AV signature, but the cost of fixing a broken model is equally expensive.
We believe that the solution lies in a hybrid approach. Using AI/ML primarily for the unknown, but verifying with tried and tested techniques used in the legacy world. This is really just another implementation of the defense in depth concept, applied to the endpoint protection world.

...and this is exactly how some of us have advocated the use of Cylance and other similar capabilities here at MT.

 

[artek]

Wouldn't the most obvious hotfix be to have the model ignore the string found in the popular video game in the good/bad score? They seemed to have added that adhoc to avoid a recurring game false postive. That doesn't seem like the most complicated thing on the planet.

Here's the quote from the article:
"Combining an analysis of the feature extraction process, its heavy reliance on strings, and its strong bias for this specific game, we are capable of crafting a simple and rather amusing bypass. Namely, by appending a selected list of strings to a malicious file, we are capable of changing its score significantly, avoiding detection."

Do that and finally let us whitelist our steam folders. Problem solved.

 

[Burrito]

Wouldn't the most obvious hotfix be to have the model ignore the string found in the popular video game in the good/bad score? They seemed to have added that adhoc to avoid a recurring game false postive. That doesn't seem like the most complicated thing on the planet.

Here's the quote from the article:
"Combining an analysis of the feature extraction process, its heavy reliance on strings, and its strong bias for this specific game, we are capable of crafting a simple and rather amusing bypass. Namely, by appending a selected list of strings to a malicious file, we are capable of changing its score significantly, avoiding detection."

Do that and finally let us whitelist our steam folders. Problem solved.

Yeah, to address the bypass in the article, you are right -- I believe.

But..... The article puts forth the implication that there could be many... hundreds or thousands of applications where malicious code could be appended to whitelisted code.

So I think the larger question is.... how does Cylance fix the algorithm that apparently universally could allow malicious coded appended to whitelisted code to subvert the malware scoring model.

Cylance seems to have taken a shortcut to avoid false positives which opened up a vector for malware attack.

I can't say this is at all surprising to me. With more research, other vectors of attack can be found... and that won't change. It just makes Cylance and other ML/AI products more like 'earlier gen' products than their marketing would like you to believe.

 

[Nightwalker]

Yeah, to address the bypass in the article, you are right -- I believe.

But..... The article puts forth the implication that there could be many... hundreds or thousands of applications where malicious code could be appended to whitelisted code.

So I think the larger question is.... how does Cylance fix the algorithm that apparently universally could allow malicious coded appended to whitelisted code to subvert the malware scoring model.

Cylance seems to have taken a shortcut to avoid false positives which opened up a vector for malware attack.

I can't say this is at all surprising to me. With more research, other vectors of attack can be found... and that won't change. It just makes Cylance and other ML/AI products more like 'earlier gen' products than their marketing would like you to believe.

Exactly, the AI engine itself is flawed because of strong bias to balance false positives, it isnt something simple that could be fixed with a signature (like an antivirus engine).

Now that Cylance has a big market share and cant use security through obscurity, expect more bypasses and exploits; this situation is something normal in this kind of industry, but the lack of updates, this characteristic advantage is now its biggest weakness, unlike hybrid solutions (the so called old antivirus) can use updates to fix this kind of problem.

ESET was triggered by Cylance marketing departament and made many good articles about AI and machine learning, specifically adressing that it isnt real AI and why it needs constant updates.

Some good articles:

ML-era in cybersecurity: A step toward a safer world or the brink of chaos?

The ML-era in cybersecurity will be an exciting period full of transformative potential to improve all areas of modern life. However, like most disruptive innovations, even ML will have its drawbacks.

www.welivesecurity.com

Don’t buy the elixir of youth: Machine learning is not magic

There is no magic in machine learning. It's a field of computer science that gives computers the ability to find patterns in huge amounts of data.

www.welivesecurity.com

Security updates belong in the limelight, not in the dustbin of history

Without regular security updates, your endpoint will be left standing alone against an entire army of cybercriminals who see you as easy prey.

www.welivesecurity.com

 

[artek]

Yeah, to address the bypass in the article, you are right -- I believe.

But..... The article puts forth the implication that there could be many... hundreds or thousands of applications where malicious code could be appended to whitelisted code.

So I think the larger question is.... how does Cylance fix the algorithm that apparently universally could allow malicious coded appended to whitelisted code to subvert the malware scoring model.

Cylance seems to have taken a shortcut to avoid false positives which opened up a vector for malware attack.

I can't say this is at all surprising to me. With more research, other vectors of attack can be found... and that won't change. It just makes Cylance and other ML/AI products more like 'earlier gen' products than their marketing would like you to believe.

Which article was that in?

I think saying that they've taken this shortcut many times is just speculation. I'm reasonably certain I know which game or which software was the culprit in the first place, because it would have to be a well known enough game from them to want to put in a global bypass, and when the consumer version of Cylance came out there was a month or so where every new version would get flagged. They even still have a general detection for pua.game, though I've only ever seen that one on very simplistic text-only based games. My guess would be if you wanted to find more bypasses like this you'd look at software which ostensibly behaves like malware but is nevertheless a legitimate file used for different purposes, but again you'd have to narrow that down with a frequently changing application that is sufficiently popular and wide spread that it was being detected on enough systems and complained about by enough people that they felt the need to do this, and I'm not sure how big of a list of files you can come up with that fit that category.

 

[ForgottenSeer 58943]

BlackBerry Cylance Guidance on Bypass Disclosure

BlackBerry® Cylance® is aware that a bypass has been publicly disclosed by security researchers. Our research and development teams have identified a solution and will release a hotfix automatically to all customers.

threatvector.cylance.com

Hotfix in few days.. I wanna see that, will Cylance once again chimp out and make it look alike it's nothing serious to worry about? As a home user I don't give a flying f, i'd just uninstall it, but if I were running a enterprise with a thousands of endpoints equiped with Cylance..

A quick hotfix, this fast, might simply be they remove the bias towards that particular game, or a string related to several games. However, that article is more damning when you pick out a few specific quotes, such as:

We can only speculate as to why this mechanism was introduced, but we believe that Cylance’s team encountered some false positives and false negatives in the main model. They could have probably adapted or improved their model but maybe they had time pressure, so the R&D team had to come up with something quick that would make the problem go away.

Which basically means, at some phase in the project, someone - marketing, sales, C level got tired of hearing complains about false positives, and rather than continue to work on the engine they put in a final override check which overrides all other malicious indicates and flags the file as benevolent, thereby reducing their entire product to a small area in the code with ultimate override power? That's a pretty significant flaw, and indicates at some point they got lazy with their product. Also that area of code could potentially make the entire AI/ML model easy to defeat once the override check parameters are discovered.

Cylance executives already cashed out, they don't care. Even their EULA prohibiting testing on the product isn't going to protect it.

However I would like to point out a coincidence that always seems to happen.. A product/company is close to selling, or gets sold and something pwns it or some grand exploit, compromise or whatever comes out. I almost have a sort of instinctive action to 'bail' anytime something is acquired by something else. I suppose these C level folks reach a 'Time to gooooooo!' point that is often precipitated by the knowledge that their days (or product days) are numbered.. Who knows.. But the quickest way for me to ditch a product is to have it acquired.

 

[Burrito]

Which article was that in?

I think saying that they've taken this shortcut many times is just speculation. I'm reasonably certain I know which game or which software was the culprit in the first place, because it would have to be a well known enough game from them to want to put in a global bypass, and when the consumer version of Cylance came out there was a month or so where every new version would get flagged. They even still have a general detection for pua.game, though I've only ever seen that one on very simplistic text-only based games. My guess would be if you wanted to find more bypasses like this you'd look at software which ostensibly behaves like malware but is nevertheless a legitimate file used for different purposes, but again you'd have to narrow that down with a frequently changing application that is sufficiently popular and wide spread that it was being detected on enough systems and complained about by enough people that they felt the need to do this, and I'm not sure how big of a list of files you can come up with that fit that category.

Nah.... that's just the way the industry and process work.

Vendors don't just whitelist a few games and / or other random files.

White lists are vast.

White lists in six-figures can be purchased from established vendors.

This article alludes to the enormity of the problem.

And don't misunderstand me.... I'm a Cylance fan. I use it and like it. The model is great. It works. They just need to go in and fix the algorithm -- which is doable but can be an arduous task.

Cylance’s AI based antivirus product can be gamed so that attackers can bypass the system’s machine learning algorithm and suspect code can be inserted from a file been previously marked as safe.

SC Media UK

www.scmagazineuk.com

A quick hotfix, this fast, might simply be they remove the bias towards that particular game, or a string related to several games. However, that article is more damning....

Yep, true dat Sly.

However I would like to point out a coincidence that always seems to happen.. A product/company is close to selling, or gets sold and something pwns it or some grand exploit, compromise or whatever comes out. I almost have a sort of instinctive action to 'bail' anytime something is acquired by something else.

Unfortunately, for US companies, this is often true.

It will be a sad day if Norton takes a winning track record of over a decade and ruin it by selling-out for a quick payday for a relative few people.

 

[Burrito]

Exactly, the AI engine itself is flawed because of strong bias to balance false positives, it isnt something simple that could be fixed with a signature (like an antivirus engine).

Now that Cylance has a big market share and cant use security through obscurity, expect more bypasses and exploits; this situation is something normal in this kind of industry, but the lack of updates, this characteristic advantage is now its biggest weakness, unlike hybrid solutions (the so called old antivirus) can use updates to fix this kind of problem.

ESET was triggered by Cylance marketing departament and made many good articles about AI and machine learning, specifically adressing that it isnt real AI and why it needs constant updates.

Some good articles:

ML-era in cybersecurity: A step toward a safer world or the brink of chaos?

The ML-era in cybersecurity will be an exciting period full of transformative potential to improve all areas of modern life. However, like most disruptive innovations, even ML will have its drawbacks.

www.welivesecurity.com

Don’t buy the elixir of youth: Machine learning is not magic

There is no magic in machine learning. It's a field of computer science that gives computers the ability to find patterns in huge amounts of data.

www.welivesecurity.com

Security updates belong in the limelight, not in the dustbin of history

Without regular security updates, your endpoint will be left standing alone against an entire army of cybercriminals who see you as easy prey.

www.welivesecurity.com

Good articles.

And ESET and others were triggered by Cylance.... and rightly so. Norton was triggered...

Home - Broadcom Community - Discussion Forums, Technical Docs, Ideas and Blogs

The site home page

www.symantec.com

When Cylance started their "Unbelievable Tour" (that's some unintended irony) -- they were duplicitous and deceptive. I hated Cylance at first.... 'cause they were a bunch of liars. They've calmed down now.... but still exaggerate their products.

And again... I like Cylance. Just none of us should be blind fanboys just because we own a product..

 

[ForgottenSeer 58943]

I like Cylance as an adjunct to other solutions. A sort of layered approach and using it as a Zero-Day. I actually don't mind some false positives if the engine benefits and is stronger as a result of it.

Cylance catches some very fishy things at times, which I appreciate. After the cash out I bolted from it but I miss it. As Burrito says, it's often common with US Companies that when they sell or approach sale shenanigans start up so I am instinctively bolting each time this happens.