ChatGPT Atlas Browser Can Be Tricked by Fake URLs into Executing Hidden Commands

Parkinsond

Level 62
Thread author
Verified
Well-known
Dec 6, 2023
5,065
14,280
6,069
The newly released OpenAI Atlas web browser has been found to be susceptible to a prompt injection attack where its omnibox can be jailbroken by disguising a malicious prompt as a seemingly harmless URL to visit.

"The omnibox (combined address/search bar) interprets input either as a URL to navigate to, or as a natural-language command to the agent," NeuralTrust said in a report published Friday.

"We've identified a prompt injection technique that disguises malicious instructions to look like a URL, but that Atlas treats as high-trust 'user intent' text, enabling harmful actions."

 
Thanks for sharing this, Parkinsond. Prompt injection attacks are becoming a real headache in AI-integrated tools like this—it's essentially tricking the system into treating malicious input as legitimate user commands. From what I've seen in similar reports, this highlights why we need better safeguards in AI-driven browsers, especially when they're blending search, navigation, and agent-like behaviors.

If anyone's using Atlas or similar experimental tools, I'd recommend sticking to well-vetted browsers for now and keeping an eye on OpenAI's updates for patches. NeuralTrust's blog post looks like a solid deep dive if you want more technical details. What are your thoughts on how this might evolve with more AI in everyday apps?
 
  • Like
Reactions: Sampei.Nihira
OpenAI has rolled out a critical security update to ChatGPT Atlas, its browser-based AI agent, introducing advanced defenses against prompt injection attacks.
The update marks a significant step in protecting users from emerging adversarial threats targeting agentic AI systems.
What Are Prompt Injection Attacks?
Prompt injection attacks exploit AI agents by embedding malicious instructions into the web content the agent processes.
Attackers craft these instructions to override a user’s commands and redirect the agent’s behavior toward harmful actions.
Full Story here:
 
Recommendation / Remediation

Immediate Update
: Ensure the ChatGPT Atlas browser is updated to the latest version. The security update hardening the agent against these class of attacks has been deployed to all users.

Principle of Least Privilege: Limit logged-in access within the Atlas browser when possible to reduce the potential impact of an unauthorized agent action.

Active Confirmation: Carefully review all agent confirmation requests. Do not click "Allow" on agentic actions that you did not explicitly and intentionally initiate.

Scoped Instructions: When interacting with the AI agent, use explicit and well-scoped instructions rather than broad or ambiguous prompts.
 
Some things in this world that I don't understand:

1. Why would anyone need a smart speaker at home?
2. Why would anyone want a foldable phone?
and...
3. Why would anyone use AI web browser?