Prompt injection in production: how agent containment actually works

When agents got tools, what actually broke?

For a while, prompt injection was a parlor trick. You told a chatbot to ignore its instructions, it said something it shouldn't, everyone screenshotted it, nothing exfiltrated. Then the assistants got mailboxes, repositories, databases, and browsers, and the same trick stopped being a trick.

The clearest turn came with EchoLeak. Disclosed by Aim Security researchers in June 2025 and tracked as CVE-2025-32711 with a CVSS score of 9.3, it was a zero-click attack on Microsoft 365 Copilot [1, 2]. An attacker sent an ordinary-looking email. The victim never had to open it. Copilot ingested it during normal background processing, and a later, unrelated user query was enough to trigger exfiltration of internal data through Copilot's own access to SharePoint, OneDrive, and chat history [1, 2]. Microsoft patched it server-side before public disclosure and reported no evidence of exploitation in the wild [1]. What made it a landmark wasn't the blast radius. It was the category. One academic write-up called it the first known real-world zero-click prompt injection in a production language-model system, and the exploit chain worked by evading Microsoft's cross-prompt injection classifier and abusing already-approved Microsoft domains to carry data out [2].

That shape repeated across the year with different logos. In Salesforce Agentforce, researchers at Noma Security found ForcedLeak, a CVSS 9.4 flaw they reported on July 28, 2025 [3]. An attacker filled in the description field of a public Web-to-Lead form with hidden instructions. When an employee later asked Agentforce about the lead, the agent followed those instructions and shipped CRM data to an attacker-controlled server. The exfiltration route was almost comic: an expired domain, still sitting on Salesforce's content-security allowlist, that the researchers bought for about $5 [3]. Salesforce enforced a trusted-URL restriction on September 8 and the finding went public on September 25 [3].

The coding tools were worse, because the agent's reach was a shell. CVE-2025-53773 let a prompt injection turn GitHub Copilot in Visual Studio Code into remote code execution [4]. The path was elegant and grim: injected text got Copilot to write to the project's `.vscode/settings.json` and flip `chat.tools.autoApprove` to true, a so-called "YOLO mode" that switched off the human confirmation step and let the agent run shell commands without asking [4]. It was reported to Microsoft on June 29 and patched in the August 2025 update [4]. Cursor had its own version, CVE-2025-54135, which Aim Labs nicknamed CurXecute, a CVSS 8.6 flaw where a single externally hosted injection rewrote the `~/.cursor/mcp.json` config and ran attacker commands without confirmation; it was reported July 7 and fixed in version 1.3 on July 29 [5, 6].

The Model Context Protocol, the standard that lets agents plug into tools, turned out to be a delivery system for the same idea. In July 2025, a team using Cursor connected to a Supabase database with a service-role key that bypasses row-level security. A support ticket contained hidden instructions; the agent read the ticket, queried an `integration_tokens` table, and wrote the secrets straight back into the ticket thread where the attacker could read them [7, 8]. Simon Willison, who documented it, kept returning to the same phrase for why it worked, which we'll come back to. Two months earlier, Invariant Labs had shown the official GitHub MCP integration doing the analog: a malicious public issue, an agent told to review the repository's open issues, and private repository contents posted back as a comment [27].

Two incidents rounded out the picture by removing the model's own judgment from the equation entirely. In July 2025, someone slipped a "wiper" prompt into the Amazon Q extension for Visual Studio Code through a pull request; the compromised version 1.84.0 shipped to a marketplace with nearly one million installs before AWS replaced it with 1.85.0 [10, 11]. The injected instruction told the assistant to clean the system "to a near-factory state" and delete cloud resources, and the person responsible later said it was deliberately broken as a warning [10, 11]. Then in August, the s1ngularity attack poisoned the Nx build package, which carries over 4.6 million weekly downloads, and did something new: the malware went looking for installed AI command-line tools like Claude, Gemini, and Q and drove them with flags such as `--dangerously-skip-permissions` and `--trust-all-tools` to scout the filesystem and exfiltrate credentials, leaking roughly 20,000 files [12]. The attacker didn't defeat the agent. They found the switch that turned its safety off and flipped it.

None of these needed a jailbreak in the old sense. They needed an agent that could read something untrusted and then reach something that mattered.

Why can't you just train the model to say no?

The obvious fix is to teach the model to recognize a malicious instruction and refuse it. Everyone tried. The results are the reason this piece exists.

OpenAI's public work is the most concrete. Its instruction hierarchy defines a trust order, System over Developer over User over Tool, so that text arriving from a tool output or a web page can't override a system policy [13]. In 2026 the company released a training dataset, IH-Challenge, and reported that fine-tuning a small model on it raised instruction-hierarchy resilience by an average of 10.0 percentage points across 16 benchmarks, from 84.1% to 94.1%, while cutting unsafe behavior from 6.6% to 0.7% [13, 14]. These are real improvements, and OpenAI is candid that they're measured on its own evaluations. They're also, read the other way, a model that still fails a meaningful fraction of the time.

Microsoft's Spotlighting takes a different tack, marking untrusted text so the model can tell data from instructions using delimiting, datamarking, or encoding. In the original experiments on GPT-family models, the authors reported the indirect-injection attack success rate dropping from over 50% to below 2%, and wrote that spotlighting "reduces the attack success rate from greater than 50% to below 2% in our experiments with minimal impact on task efficacy" [15]. That's a striking number. It also comes from a 2024 paper run against 2023-era models, which is worth holding in mind when it gets cited as a current guarantee, and even there the technique cut the rate sharply but not to nothing [15].

Google's posture, published in June 2025, is the most honest about what a single technique buys you: not much, so use several. Its layered approach for Gemini stacks adversarial training into the model, purpose-built classifiers to filter malicious instructions out of Workspace content, a "security thought reinforcement" prompt that re-reminds the model to ignore embedded instructions, markdown sanitization to block image-based exfiltration, and user-confirmation prompts for risky actions [16]. Google notes that its Gmail defenses already block more than 99.9% of spam, phishing, and malware, and that EchoLeak's specific trick didn't apply to Gemini because a markdown sanitizer blocked the external image render [16]. The framing is telling. Google isn't claiming the model won't be fooled. It's claiming that when the model is fooled, the exfiltration channel is already closed.

The reason none of this converges on zero is structural, and the incident reports keep saying it in the same words. A language model reads instructions and data in one undifferentiated stream, so an attacker who controls any of the data can try to promote it to an instruction. The EchoLeak analysis called indirect injection structurally harder than direct injection precisely because the malicious text rides in on content the assistant was supposed to process [1]. The Supabase write-up put the operational version bluntly: security controls can't be reliably enforced inside the model, because the model is stochastic, so the enforcement has to live somewhere deterministic [8]. That's not a counsel of despair. It's a redirection. If the model can't be trusted to refuse, the job moves to whatever sits between the model's decision and the world.

If not the model, then what holds?

The single most useful idea to come out of the last year is a way of naming the danger so you can see it before it fires. Simon Willison called it the lethal trifecta: an agent that has access to private data, exposure to untrusted content, and a way to communicate externally will, sooner or later, leak [7, 17]. The insight is that no one leg is the vulnerability. The exposure is the combination. An agent that only reads untrusted content but can't reach anything sensitive is a curiosity. An agent that holds secrets but never sees attacker-controlled text is a normal program. Put all three together and you've built an exfiltration machine.

Read the incidents through that lens and they stop looking like separate bugs. ForcedLeak was CRM data (private) plus a public web form (untrusted) plus an allowlisted domain (egress) [3]. Supabase was a privileged database key (private) plus a support ticket (untrusted) plus a writable public table (egress) [7, 8]. The ForcedLeak post-mortems reached for the same frame, noting that the content-security policy failed because it checked destination URLs rather than the legitimacy of the agent's outbound request, which is another way of saying the egress leg was left intact [3].

The design conclusion writes itself: break a leg on purpose. Don't let one agent both read untrusted content and hold privileged tools. That principle is why the research frontier has quietly abandoned making the model safer for making the system safe around an unsafe model. The clearest expression is CaMeL, a design from a Google-affiliated team including Debenedetti, Carlini, and Tramèr [18]. It splits the work in two: a privileged model turns the trusted user request into a plan, and a separate quarantined model handles the untrusted data with no tool access at all, while a custom interpreter tracks where every value came from and enforces a capability policy before any tool call fires [18]. Because the untrusted data literally cannot influence control flow, the guarantee, within its threat model, is structural rather than probabilistic. On the AgentDojo benchmark, the paper reports CaMeL solving 77% of tasks with provable security against 84% for an undefended agent [18]. That gap, 7 points of utility for a security property you can actually reason about, is the real price of containment stated in one number, and it's a design-stage result on a benchmark, not a shipped product.

CaMeL isn't alone, and that matters more than any single design. DRIFT, presented at NeurIPS 2025, enforces control-level and data-level constraints through a secure planner, a dynamic validator, and an injection isolator, and reports strong security while keeping utility on AgentDojo [19]. Different mechanics, same conviction: put a deterministic policy between the model and its tools. When independent groups keep arriving at the same architecture from different directions, that's a signal the field has found the load-bearing idea. It is not yet a signal that the idea ships at scale, and the honest reading is that these are graded designs, promising and unproven in the wild, not production track records.

What does the containment stack look like in production?

Strip the research vocabulary away and containment in production is a small number of boring, verifiable controls. Each one assumes the injection already succeeded and asks what the compromised agent can actually do.

The first control is permission and tool gating: the agent gets the narrowest set of tools and scopes that let it do its job, and nothing more. The incidents are a catalog of what happens without it. The Supabase agent ran with a service-role key that bypassed row-level security, so a support ticket could read a secrets table [7, 8]. The Amazon Q and Nx attacks turned on `--trust-all-tools` and `--dangerously-skip-permissions`, flags whose entire purpose is to remove the gate [11, 12]. OWASP's 2025 list foregrounds Excessive Agency (LLM06) precisely to name this: granting an agent capabilities beyond its task creates a proportionally larger blast radius when it's compromised [20].

The second control is egress: constrain where the agent can send data so that a successful injection has nowhere to ship it. This is the control that turns a catastrophe into a near-miss, and Salesforce's fix is the cleanest case study. ForcedLeak worked because an expired allowlisted domain gave the agent a trusted outbound channel, and Salesforce's remediation was to enforce a trusted-URL restriction so agents couldn't send output to untrusted destinations [3]. Google's markdown sanitizer is the same idea at a different layer, blocking the image-render trick that EchoLeak-style attacks use to smuggle data out through a URL [16]. Egress control is unglamorous and it's the leg most often left intact, which is exactly why it keeps being the leg that leaks.

The third control is human confirmation, but tied to specific high-risk actions rather than sprinkled everywhere. A confirmation prompt is worthless if users click through it a hundred times a day, and it's the whole game if it fires only on money movement, external messaging, code execution, or permission changes. The coding-tool CVEs are what happens when this control is absent or removable: CurXecute executed commands from new `mcp.json` entries without confirmation, and the Copilot RCE worked by writing a setting that disabled confirmations entirely [4, 6]. Anthropic's browser pilot leans hard on this control, requiring site-level permissions and action confirmations for high-risk steps and blocking whole categories of sites like financial services outright [21]. Its computer-use feature ships with a denied-apps list so users can keep the agent out of password managers and banking apps before it ever starts [22].

The fourth control is sandboxing and isolation: run the agent where a compromise can't reach the rest of the machine or persist. Confine filesystem access to a working directory, deny child processes, spin up fresh environments per task so a poisoned state doesn't survive. The value shows up in the negative space of the incidents, where the agent ran with the user's full privileges across authenticated sessions, which is exactly how the browser attacks bypassed the same-origin protections that keep normal web pages apart [23].

The uncomfortable truth underneath the stack is that every layer assumes the one above it failed. That's not redundancy for its own sake. It's the only posture consistent with a model you can't stop from being fooled.

Whose posture actually holds up?

Vendors don't disclose the same things, so a fair comparison has to grade on posture and candor rather than a single leaderboard. What follows separates how well-documented each claim is from how much it should reassure you.

The best-documented incidents are the CVEs with named researchers, patch dates, and multiple independent write-ups: EchoLeak, ForcedLeak, the Copilot and Cursor RCEs, the Supabase and GitHub MCP leaks, Amazon Q, and Nx [1, 3, 4, 5, 7, 10, 12, 27]. Treat those as solid. The defense numbers are a weaker tier, because they're almost all self-reported on the vendor's own evaluation, and the research designs are weaker still as evidence about production, because they're graded on benchmarks and interesting rather than proven.

Anthropic's browser posture is the most instructive because it's the most transparent about failing. Piloting Claude for Chrome, the company reported that its autonomous mode had a prompt-injection attack success rate of 23.6%, which its mitigations reduced to 11.2% [21, 24]. In its own words: "When we added safety mitigations to autonomous mode, we reduced the attack success rate of 23.6% to 11.2%, which represents a meaningful improvement over our existing Computer Use capability" [24]. On a narrower set of browser-specific attacks like hidden form fields, new mitigations took the rate from 35.7% to 0% [21]. The number to sit with is 11.2%, which Anthropic itself frames as not good enough for wide release, and the containment story around it (site permissions, action confirmations, category blocks) is doing the real work [21, 24].

Microsoft and Google both run layered stacks, and both got their public tests. Microsoft's Spotlighting research shows the dramatic laboratory drop, over 50% to below 2% [15], but that's a 2024 result and Microsoft also lived through EchoLeak in a shipped product [1], a reminder that a good lab number and a secure deployment are different achievements. Google's layered defense is the more current articulation and the most explicit that classifiers and training are only part of a stack that leans on egress sanitization and confirmation to actually contain a failure [16].

Salesforce and the browser vendors sit at the two ends of the response spectrum. Salesforce shipped a containment fix, trusted-URL enforcement, that directly closed the egress leg ForcedLeak abused, which is roughly the right lesson learned the hard way [3]. The consumer AI browsers are the cautionary case. In LayerX's testing of 103 real-world phishing and web attacks at the time of publication, ChatGPT Atlas stopped 5.8% of malicious pages while Chrome stopped 47% and Edge stopped 53%, and separate LayerX testing put Perplexity's Comet at 7% [23, 25]. Those are third-party numbers, not vendor claims, and they describe browsers that inherit the user's full privileges the moment an injection lands. By December 2025, one advisory cited by an enterprise-hardening guide went as far as recommending organizations block AI browsers outright, warning that data lost to them "can be irreversible and untraceable" [25]. That's a market telling you the containment isn't there yet.

OpenAI's disclosed posture is the most model-centric of the group, resting on instruction-hierarchy training that improves the numbers without claiming to close the gap [13, 14]. As a research result it's strong. As an operator's guarantee it's the same caveat as everyone else's: better odds, not a wall.

What should an operator require before shipping an agent?

If you're the person actually deploying an agent, the year's lesson collapses into a short list of things to demand, each mapped to a leg you're trying to break.

Require least privilege as a hard scope, not a policy. The agent should hold the narrowest tool set and the tightest credentials that let it finish its task, and you should be able to name what it can't reach. The service-role key in the Supabase incident and the `--trust-all-tools` flags in the Amazon Q and Nx attacks are the anti-patterns; if your agent runs with a superuser credential or a trust-everything flag, you've pre-approved the breach [7, 11, 12]. OWASP's Excessive Agency entry exists to give this requirement a name in a risk register [20].

Require egress control you can point to. There should be an allowlist of destinations the agent can send data to, enforced deterministically, and it should evaluate the request rather than trust a domain that once looked fine. ForcedLeak is the whole argument: a stale allowlisted domain was the difference between a blocked injection and exfiltrated CRM data [3].

Require confirmation gates bound to risk classes. Human approval has to fire on the actions that cause irreversible harm, money movement, external messaging, code execution, permission elevation, and it has to be un-disableable by the agent. The Copilot RCE is the counterexample that proves the rule, since the exploit's core move was writing a config that turned confirmations off [4].

Require sandboxed, ephemeral execution. The agent runs confined to a working directory, without the ability to spawn arbitrary processes or persist across tasks, so a compromise is contained to one run and one scope. The browser attacks that rode the user's full authenticated session are what the absence of this looks like [23]. Ephemerality also blunts persistent memory poisoning: LayerX showed instructions planted in ChatGPT's memory through cross-site request forgery persisting across sessions, devices, and browser reinstalls, a foothold that per-task state denies [23].

Require provenance and separation for untrusted input. The system should track where each piece of content came from and keep untrusted data from steering privileged actions, which is the shippable shadow of what CaMeL and DRIFT do in the lab [18, 19]. You may not have a formal interpreter, but you can at least refuse to let the agent that reads a support ticket be the same agent that holds the database key.

Require logs that reconstruct the decision. When something goes wrong, you need to know what the model saw, what was approved, and which tool executed which step. Several of the year's incidents were caught or understood only because someone could trace the agent's actions after the fact, and the MCP git flaws were notable precisely because, in the words of one researcher quoted by The Hacker News, an attacker only needs to influence "what an AI assistant reads (a malicious README, a poisoned issue description, a compromised webpage)" to weaponize them [9]. Pair those logs with runtime monitoring that evaluates each tool call and output before it fires, since a deterministic check on the agent's actions can intercept an injection chain that pre-deployment review misses [3, 26]. If you can't see what your agent read, you can't tell an injection from a bad day.

None of these require you to solve prompt injection. They require you to assume you won't.

Who benefits, and what would prove this wrong?

It's worth asking who gains from the story this piece just told, because a clean narrative is a thing to be suspicious of. Security vendors benefit from "the model can't be trusted, buy a control plane," and the containment framing sells gateways, runtime monitors, and enforcement layers. The startup that disclosed the Copilot Studio flaw, CVE-2026-21520, a CVSS 7.5 indirect injection patched by Microsoft on January 15, 2026, exited stealth with a $7 million round and its own open-source enforcement product, which doesn't make the finding wrong but does explain the enthusiasm [26]. Model vendors, for their part, benefit from the opposite story, that hardening is working and the numbers are trending down, because it keeps agents shippable. Both incentives are real and both produce true statements, which is why the percentages here are badged and the head-to-heads are missing.

The evidence has honest weaknesses. Nearly every defense figure is a self-reported before-and-after on the reporter's own test set, and there is no independent, standardized head-to-head that would let you say one vendor's containment beats another's. A benchmark result like CaMeL's 77% is a static score, and static scores flatter defenses because the attacker in the benchmark isn't adapting to the defense the way a real one would [18]. The browser numbers are third-party, which is better, but they measure phishing-page blocking rather than the full space of injection, so they're a proxy [23, 25]. And a trend across nine months of incidents is not a law; that indirect injection kept beating model-level defenses through mid-2026 doesn't guarantee the next model generation won't shift the balance.

So what would falsify the containment thesis? Two things, mainly. If a model-only defense, no architecture, no egress control, just training, held at a near-zero attack success rate under a genuinely adaptive red team and an independent one, that would be evidence the problem is tractable at the model layer after all, and the whole "box it in" posture would look like over-engineering. Nothing in the past year's disclosures comes close. The other direction is worse: if a containment architecture with provable-security claims were shown to leak anyway through a channel the policy didn't model, the "structural guarantee" language would need heavy qualification. The MCP git-server flaws, where an attacker chained a poisoned README through a filesystem tool to reach code execution, are a small preview of how a compromise routes around a boundary that only watched the obvious door [9].

The uncomfortable place to leave it is that no one deploying agents today gets to claim their system can't be injected. That claim is barred by the physics of a model that reads instructions and data in the same breath. What an operator can claim, and should be made to prove, is narrower and more useful: that when the injection lands, and it will, the agent is holding nothing worth stealing, pointed at nowhere worth sending it, and stopped at the one door that matters until a human says otherwise.