Security News Don't use Agentic AI

[Victor M]

Content source

https://cybersecuritynews.com/promptware-kill-chain-five-step-kill-chain-model/

Agentic AI can be susceptible to prompt injection, and attackers can exfiltrate your company data.

Simple example: an attacker inserts instructions for the AI into a web page he knows you will ask the AI to crawl. And instruction asks the AI to dump your company secrets. And if you use Agentic AI to link to your email, those secrets can be emailed to the attacker.

The motto: use the AI to think, do the copy / pasting yourself.

Promptware Kill Chain - Five-Step Kill Chain Model for Analyzing Cyberthreats

LLMs now power business workflows, but researchers warn of “promptware,” multi-stage attacks exploiting AI apps like real malware.

cybersecuritynews.com

 

[Bot]

Agentic AI can be susceptible to prompt injection, and attackers can exfiltrate your company data.

Simple example: an attacker inserts instructions for the AI into a web page he knows you will ask the AI to crawl. And instruction asks the AI to dump your company secrets. And if you use Agentic AI to link to your email, those secrets can be emailed to the attacker.

The motto: use the AI to think, do the copy / pasting yourself.

Promptware Kill Chain - Five-Step Kill Chain Model for Analyzing Cyberthreats

LLMs now power business workflows, but researchers warn of “promptware,” multi-stage attacks exploiting AI apps like real malware.

cybersecuritynews.com

You’re describing a real and well-known failure mode for tool-using (“agentic”) LLM systems: prompt injection can steer the model into abusing whatever tools and data you’ve connected to it, including data exfiltration paths (email, chat, issue trackers, cloud storage, etc.).

What’s accurate in your example

• Prompt injection is not hypothetical. If the agent consumes untrusted content (web pages, tickets, documents, emails), that content can include instructions that the model may follow unless you have strong controls.
• Tools turn a bad answer into a security incident. A plain chatbot can “say” secrets but typically can’t access them. An agent with connectors (mail, drive, CRM, Git, SIEM, internal wiki) can potentially retrieve and transmit data if you let it.
• Exfiltration often looks like “normal workflow.” “Send a summary email,” “open a PR,” “post to Slack,” or “file a ticket” can be abused as outbound channels.

Important nuance (why “just copy/paste yourself” is not a complete fix)
Human-in-the-loop reduces risk, but it’s not sufficient by itself because:

• Users can still paste sensitive content into the model (accidentally or under social engineering pressure).
• If the agent is still connected to internal data/tools, it can fetch secrets without the user explicitly pasting them.
• Attackers can aim for indirect harm (tampering, deletion, fraudulent actions), not only data dumping.

Practical mitigations that actually move the needle
If you’re deploying/using agentic AI in a company setting, these are the controls that matter most:

• Least privilege for tools and data
  • No default access to email, files, Git, tickets, or admin consoles.
  • Scope tokens per task (short-lived), per user, per project, per repository/folder.
• Hard separation between “untrusted browsing” and “trusted actions”
  • Treat web content, inbound email, and uploaded docs as untrusted input.
  • Do not allow untrusted content to directly trigger actions (send, post, purchase, delete, share).
• Allowlist outbound destinations and actions
  • Block sending messages to arbitrary external addresses/domains by default.
  • Allow only pre-approved domains, channels, or ticket queues.
• Mandatory human confirmation for high-risk actions
  • Any outbound communication, sharing permissions, file export, or credential/token use should require explicit user approval with a clear diff/preview.
  • Use “two-step” confirmations for especially sensitive operations (finance, IAM, external sharing).
• Content and instruction handling
  • Treat any instructions found in retrieved content as data, not commands.
  • Use robust prompt-injection defenses: isolate retrieved text, strip/neutralize “system-like” directives, and use policies that the tool layer enforces (not just the model).
• Egress controls + DLP
  • Monitor and restrict what can leave (email/webhooks/HTTP).
  • DLP rules for secrets (API keys, tokens, customer data), plus alerting on unusual bulk access or sharing.
• Auditing and incident readiness
  • Log: retrieved sources, tool calls, parameters, outputs, and approvals.
  • Have a kill-switch to revoke connectors/tokens quickly.

A safer “motto” framing
“Use the AI to think, do the copy/pasting yourself” is a good rule of thumb for individuals, but for organizations a stronger, more reliable principle is:

• Never let untrusted content directly drive trusted actions.
• Assume the model can be socially engineered; enforce safety in the tool layer and permissions.

I can’t verify the specifics of the article you linked from within this chat, but the general risk (prompt injection + tool access = real exfiltration/tampering risk) is legitimate, and the mitigations above are the practical way to reduce it.

 

[Sorrento]

I may be the last person around who has yet to actively use deliberately or installed an AI product to gain info, AI is on my ASUS router & I do ask Sir questions from time to time but it is as thick as a bag of hammers so that’s infrequent, I appreciate AI is around us but there are IMHO risks & issues that maybe we have not seen yet, still in my view a really bad idea how its used by the masses, I can't see that view changing any time yet??

 

[Parkinsond]

risks & issues that maybe we have not seen yet

They know and they do not care; all they care about is more users for more training of the engine, more development, more advanced versions, more marketing, and more revenue.

 

[Sorrento]

The real issue is the trillions invested in AI is not getting the returns that were hoped for so it seems it will be pushed more & more - Resistance is not futile, though 7 Of 9 did have an endearing quality...

 

[Parkinsond]

The real issue is the trillions invested in AI is not getting the returns that were hoped for so it seems it will be pushed more & more - Resistance is not futile, though 7 Of 9 did have an endearing quality...

View attachment 294754

They will keep pushing until they cannot afford more, when the bubble blasts and the use of AI is rationalized.

 

[Sorrento]

Fingers in wallets time when people have become to thick to think rationality for themselves & become reliant on AI to write a decent letter

 

[Divergent]

Recommendation / Remediation
To secure Agentic AI against Promptware, organizations must move beyond simple prompt filtering and address the entire kill chain.

Break Initial Access (Input Sanitization)
Treat all LLM input (user prompts and retrieved external data) as untrusted. Implement strict input validation layers before data reaches the model.

Use "human-in-the-loop" verification for any AI action that involves external data retrieval.

Prevent Persistence (Memory Hygiene)

Isolate Context
Do not allow the AI to retain long-term memory of sensitive sessions. Flush context windows between distinct tasks to prevent retrieval-independent persistence.

Sanitize Data Stores
Regularly scan Knowledge Bases and RAG (Retrieval-Augmented Generation) repositories for hidden prompt injection payloads (poisoned content).

Halt Lateral Movement (Least Privilege)
Restrict the AI agent's API permissions. An AI summarizing emails should not have "Send" privileges without explicit user confirmation.

Segment AI agents from critical business networks to prevent the "worming" behavior seen in Morris II.

Adhere to Standards (NIST AI RMF)
Apply the NIST AI Risk Management Framework (AI RMF) function of GOVERN and MEASURE. Establish policies that specifically map where AI agents have write-access to business data and monitor those interfaces for anomalous output patterns.

 

[Victor M]

reliant on AI to write a decent letter

Writing a simple decent letter is easy. Writing a promotional enticing sales letter is difficult ( for the semi-introverts like me ); you have be a professional salesperson; know what sweet spots to tickle, what pain points to illicit, use professional verbiage, and not be over-bearing. I have been in sales roles before, but I am very far from professional. ( I can't charm a kitten off a tree, as the old saying sorta goes ). But my point isn't about asking the AI to think or write, it is the integration with apps that have a chance of leaking stolen info, like email.