Malware News Fooling AI Agents: Web-Based Indirect Prompt Injection Observed in the Wild

Khushal

Level 15
Thread author
Verified
Top Poster
Well-known
Apr 4, 2024
738
4,615
1,369
Large language models (LLMs) and AI agents are becoming deeply integrated into web browsers, search engines and automated content-processing pipelines. While these integrations can expand functionality, they also introduce a new and largely underexplored attack surface. One particularly concerning class of threats is indirect prompt injection (IDPI), in which adversaries embed hidden or manipulated instructions within website content that is later ingested by an LLM. This article shares in-the-wild observations from our telemetry, including our first observed case of AI-based ad review evasion.

 
Executive Summary

Confirmed Facts

Threat actors are actively deploying Indirect Prompt Injection (IDPI) payloads within the HTML, CSS, and Javascript of live websites to manipulate the outputs of Large Language Models (LLMs) and AI agents. Telemetry indicates these attacks utilize visual concealment (e.g., font-size: 0px) and obfuscation to bypass human detection while executing high-impact intents such as SEO poisoning, unauthorized financial transactions, and AI ad review evasion.

Assessment
The structure of these attacks suggests a critical architectural vulnerability in how Agentic AI systems parse web data; LLMs currently lack the innate ability to distinguish between authoritative system prompts and untrusted web telemetry.

Technical Analysis & Remediation

MITRE ATT&CK Mapping

T1190

(Exploit Public-Facing Application - applied to LLM interfaces)

T1562.001
(Impair Defenses: Disable or Modify Tools - AI content filters).

CVE Profile
N/A (Architectural/Logic Flaw)
CISA KEV Status: Inactive.

Telemetry

Hashes

Insufficient Evidence.

IPs
Insufficient Evidence.

Domains
reviewerpress[.]com
cblanke2.pages[.]dev
llm7-landing[.]pages[.]dev.

Commands/Signatures
rm -rf --no-preserve-root, :(){ :|:& };: (Fork Bomb).

Constraint
Because raw binary analysis is unavailable, we must state the structure resembles a semantic bypass of LLM parsing logic, utilizing techniques like homoglyph substitution, payload splitting, and dynamic DOM injection to execute commands.

Remediation - THE ENTERPRISE TRACK (NIST SP 800-61r3 / CSF 2.0)

GOVERN (GV) – Crisis Management & Oversight

Command
Establish AI acceptable use policies explicitly prohibiting autonomous Agentic AI from executing high-privilege actions (e.g., financial transactions, database deletions) based on parsed web inputs.

DETECT (DE) – Monitoring & Analysis

Command
Implement robust web-filtering and content-inspection proxies to detect obfuscated prompt patterns, zero-sized text elements, and Base64-encoded strings dynamically assembled in the DOM.

RESPOND (RS) – Mitigation & Containment

Command
Suspend or isolate API keys and OAuth tokens for any AI agent exhibiting anomalous behaviors, such as initiating unauthorized authentication flows.

RECOVER (RC) – Restoration & Trust

Command
Validate the integrity of all data pipelines and databases accessed by the compromised AI agent, reverting any unauthorized modifications.

IDENTIFY & PROTECT (ID/PR) – The Feedback Loop

Command
Harden AI architecture by enforcing strict prompt sanitization, separating system instructions from untrusted data context, and mandating "human-in-the-loop" authorization for critical state changes.

Remediation - THE HOME USER TRACK (Safety Focus)

Priority 1: Safety

Command
"Disconnect from the internet immediately" is NOT required unless a locally hosted AI agent with system-level execution permissions has actively ingested the malicious payload.

Command
Do not rely solely on AI-generated summaries for critical financial, security, or purchasing decisions.

Command
Do not log into banking/email if redirected by an AI assistant until the destination URL is verified clean.

Priority 2: Identity

Command
Review and revoke OAuth grants or account connections provided to third-party AI assistants or browser plugins.

Priority 3: Persistence

Command
Audit browser extensions; immediately remove AI summarization tools or web-scrapers that request excessive permissions to read and alter all website data.

Hardening & References

Baseline

CIS Benchmarks (Web Browser Security / Least Privilege application for APIs).

Framework
NIST Artificial Intelligence Risk Management Framework (AI RMF 1.0) / NIST CSF 2.0.

Guidance
To mitigate web-based IDPI, defenders require proactive, web-scale capabilities to detect IDPI strings and distinguish benign web elements from malicious semantic prompts prior to LLM ingestion.

Source

Palo Alto Networks
 
  • Like
Reactions: harlan4096
When talking about artificial intelligence and security, someone should remember that the biggest surprise doesn’t always come from a hoodie‑wearing hacker… sometimes it’s a hidden paragraph on a webpage that turns the agent into its own accomplice. 🤖📜🙃
 
  • Like
Reactions: Brownie2019