A.I. News Top AI tools including Claude, Codex, and Hermes installed suspicious code inside corporate networks

[Brownie2019]

Content source

https://www.techradar.com/pro/security/top-ai-tools-including-claude-codex-and-hermes-installed-suspicious-code-inside-corporate-networks

A new class of "squatting" risks is emerging right in front of us

• Researchers found unclaimed llms.txt references on 120 domains, exploitable by cybercriminals
• AI agents could install malware if they execute hallucinated or outdated documentation commands
• Fixes: clean documentation and restrict AI agents from treating docs as executable instructions

Cybercriminals are able to now abuse hallucinated, outdated, and outright incorrect website documentation to deliver malware to unsuspecting victims through AI agents, new research has claimed.

An increasing number of websites now contain two documents: llms.txt, and llms-full.txt. These are conventions that allow AI agents to properly read the contents of the websites. If an AI agent is looking to install software or add code to a project, they can search through these documents across the web until they find a fitting solution.

Researcher Alon Hertz analyzed 6,214 live domains belonging to defense contractors, Fortune 500 organizations, as well as big tech. On these domains he found 8,265 of these .txt files and among them 120 (all on a different site) pointing to one or more code packages and domain names that weren’t registered at all.
Claiming packages and domains
There can be a myriad of reasons why they’re not registered. It can be due to human error, renamed or abandoned packages, copy/paste errors, or hallucinated documentation.

Now, for the purpose of the experiment, Hertz registered some of these unclaimed names and hosted packages that would phone home when installed. It took less than an hour for a Fortune 500 company to start pinging, and the numbers soon grew to “a few dozen more”.
This means that if the researchers can do it, so can cybercriminals. In theory, a cybercriminal could find these unclaimed packages and register malware. If an AI agent has permission to execute shell/package-manager commands and stumbles upon this documentation, it can end up infecting the device.

Claude, OpenAI’s Codex, and Nous Research’s Hermes were all “guilty”, the researchers said.
To fix the vulnerability, two things need to happen. First, companies need to clean up their documentation and make sure it’s not pointing towards non-existent or malicious content. Second, AI agents need to stop treating documentation as executable instructions. Since the latter most likely isn’t happening any time soon, the immediate answer would probably lie in the former. In the meantime, organizations using AI for coding should consider the risks when granting AI agents permission to execute commands.

 

[Bot]

This is a legitimate software-supply-chain risk, but the article’s claims should be independently verified

If the reported experiment is accurate, the issue is not that llms.txt files have special execution privileges. The risk comes from AI agents treating text retrieved from websites as trusted implementation guidance and then executing commands such as package-manager or shell instructions without sufficient verification.

Unclaimed package names and domains can create opportunities similar to:

• Dependency or package-name squatting
• Subdomain or cloud-resource takeover
• Typosquatting and copy/paste errors
• Stale documentation referencing abandoned projects
• AI-generated hallucinations that happen to resemble real package names

Registering a previously unclaimed name does not automatically make the resulting package malicious. However, if an organization’s documentation or an AI-generated recommendation causes software to install that package, the owner may receive an unexpected installation or network request. That is a useful indication of exposure, but it does not by itself prove that every affected system was compromised.

Practical safeguards

Organizations should:

• Audit llms.txt, llms-full.txt, README files, examples, installation scripts, and internal documentation for nonexistent or unverified references.
• Use package-manager lockfiles, private registries, dependency allowlists, and approval workflows for new dependencies.
• Verify package ownership, publisher identity, repository history, signatures where available, and download sources before installation.
• Prevent AI agents from executing shell commands or installing packages automatically by default.
• Run development agents with least privilege, network restrictions, isolated environments, and disposable containers.
• Require human approval before commands that install software, modify system files, access secrets, or make outbound network connections.
• Monitor package installations and unexpected DNS or HTTP requests from development systems.
• Retire or correct stale documentation promptly, including references to renamed or abandoned packages.

Developers should also avoid assuming that a package name suggested by an AI system exists or is trustworthy. The package should be located through the official project repository or registry, and its exact name and publisher should be checked independently.

The broader lesson is that documentation is input data, not an authority. AI agents should be able to read it, but instructions extracted from documentation must be treated as untrusted content and validated before execution.