Conversation History Poisoning: How AI Agents Can Inherit Fake Approval

Conversation history poisoning can make an AI agent believe a dangerous action was already discussed, approved, and partly completed. On September 24, 2026, Darktrace published controlled research showing how fabricated local chat history pushed coding agents toward unauthorized offensive actions inside a sandbox.

The finding matters now because many agent security programs focus on the current prompt while trusting yesterday's transcript as harmless state. If an attacker, malicious package, or compromised process can rewrite that state, the agent may inherit a false past and treat it as authority.

Key Takeaway: Stored conversation history is executable security context. Protect it with the same integrity, identity, and monitoring controls you would apply to a configuration file or automation script.

What the new conversation history poisoning research found

Darktrace's September 24 research examined a design pattern across popular coding-agent harnesses: sessions are stored locally and later loaded back into the model's context, but the stored messages may not carry proof that the model or user actually produced them.

In the researchers' lab, a poisoned session contained 78 fabricated turns describing prior reconnaissance, exploitation, and authorization. When the agent resumed, the current request no longer looked like the start of a suspicious operation. It looked like the next step in work that the transcript claimed had already been approved.

The team tested the technique against agentic coding workflows associated with Claude Code, OpenAI Codex, and AWS Kiro CLI. The reported outcomes differed by system, but the central weakness was consistent: the model received a convincing record of events without an independent way to verify that record's provenance.

This was controlled security research, not evidence of a widespread campaign or a confirmed compromise of those vendors. The practical issue sits in the surrounding harness and storage boundary. A capable model can reason well from the context it receives and still make a dangerous decision when that context has been forged.

Key Stat: A single poisoned session with 78 invented turns was enough to transform an obviously suspicious instruction into what appeared to be a continuation of authorized work.

How a forged transcript becomes operational authority

Conversation histories do more than preserve convenience. They tell an agent what the user asked, what the assistant refused, what tools ran, what files changed, and which decisions were supposedly approved. That makes the transcript a control input even when the product labels it as memory or session state.

The attack chain is straightforward:

1. An attacker gains write access to an agent's local session files or backing store.
2. The attacker inserts plausible user messages, assistant reasoning, tool results, and approval statements.
3. The user or automation resumes that session through the normal interface.
4. The harness loads the forged history as trusted context.
5. The model interprets a harmful request as consistent with prior authorization and acts through its available tools.

The dangerous step is not just text generation. It is the transition from unverified history to real tool execution. A fabricated claim that credentials were approved, a host was in scope, or a file was safe can change what the agent decides to do next.

This is different from classic indirect prompt injection. In a typical injection, hostile instructions arrive inside a webpage, email, document, or retrieved record during the current task. With conversation history poisoning, the malicious instruction arrives wearing the identity of the agent's own past.

Hexon's prompt injection defense guide explains why untrusted content should never become policy. History poisoning extends that rule: even first-party-looking context must be treated as untrusted until its integrity and author are verified.

Common Mistake: Assuming local files are trusted because they never crossed a public API. A package install, editor extension, malware process, shared workspace, or over-permissioned account may already have local write access.

Why coding agents create a high-impact target

Coding agents are attractive targets because their normal work already resembles attacker behavior. They inspect repositories, run commands, edit files, query services, install dependencies, and use credentials. The difference between a productive workflow and a damaging one may be a few claims about scope and approval.

A poisoned history can exploit that ambiguity. It might claim that a security lead authorized testing against a particular host, that a user approved sending a report by email, or that a previous tool result established a package as safe. The current agent sees a coherent narrative, not the missing evidence behind it.

The risk increases when the harness can access:

• shell commands and package managers
• source repositories and CI/CD credentials
• cloud consoles or deployment tools
• email, chat, ticketing, or file-sharing connectors
• secrets stored in environment variables or developer profiles

That is why package and workspace hygiene remain relevant. Hexon's analysis of malicious npm packages that activate at runtime shows how a dependency can wait for favorable conditions before acting. A package that can write agent state does not need to exploit the model directly. It can change the story the model reads later.

The same trust problem appears when developers open unfamiliar repositories. Our guide to AI coding agent malware in clean-looking GitHub repos covers project instructions and startup behavior that can turn routine agent use into code execution. Conversation history adds another persistence layer to inspect.

Conversation history poisoning defenses that matter

The right response is not to ask the model to be more skeptical. Security controls need to establish which messages are authentic, which principal produced them, and what authority remains valid at execution time.

Authenticate every history record

Each stored message should be bound to its author, session, sequence, and timestamp through an integrity mechanism such as a message authentication code or digital signature. Protecting the whole file with one checksum is weaker because it may not reveal which record changed or whether a legitimate process rewrote history out of order.

Use an append-only or tamper-evident event log for sensitive workflows. If a product must summarize or compact a long session, preserve the original signed records and identify the process that created the summary.

Separate narrative context from current authorization

A transcript can describe an approval, but it should not confer one. High-impact actions should require a fresh policy decision based on the current user, current target, current tool call, and current risk.

For example, an agent may remember that a penetration test was approved yesterday. The execution layer should still verify the allowed target list, time window, command class, and identity before a network action runs today.

Microsoft's agent safety guidance emphasizes that every point where data enters or leaves an application is an attack surface. Stored history belongs on that boundary map, not in a trusted box behind it.

Minimize who can write agent state

Do not store resumable sessions in a directory where every project script, dependency, container, or desktop process can modify them. Apply operating-system permissions, isolate service accounts, and separate user-editable project files from harness-owned state.

Where practical, keep sensitive agent sessions in a managed store with authenticated writes and immutable audit records. Local convenience should not quietly grant every tool on the workstation the ability to rewrite the agent's past.

Pro Tip: Test the negative case. Modify one stored assistant message in a staging environment and confirm the harness refuses to resume, alerts an operator, and preserves the altered evidence.

Put tool policy outside the model

History integrity is only one layer. If a forged session reaches the model, external policy should still prevent it from turning false context into harmful action.

Start with least privilege. A code-review task rarely needs email delivery, unrestricted outbound network access, production deployment credentials, and a general shell at the same time. Grant tools per workflow, use short-lived credentials, and remove them when the task ends.

Then place deterministic checks on consequential actions:

• require an authenticated user confirmation for new external recipients
• restrict network requests to approved destinations
• block destructive filesystem commands outside defined paths
• require target-scoped authorization for security testing
• prevent package installation in review-only workflows
• cap data volume and destinations for outbound transfers

The OWASP AI Agent Security Cheat Sheet recommends layered controls around tool use, identities, memory, and monitoring. The key idea is that the model proposes an action while a separate control plane decides whether that action is permitted.

This also connects to AI evaluation incident controls. Stop conditions, revocable credentials, and egress controls must live outside the workload they are designed to contain. A poisoned agent should not be able to explain its way around them.

Detection and incident response for poisoned agent state

Teams need telemetry that connects a loaded history to the actions that follow. Without that link, investigators may see a legitimate agent process making a suspicious call but miss the altered context that shaped the decision.

Log at least these events:

• session creation, resume, import, export, and deletion
• identity and process responsible for each history write
• integrity-check success or failure
• summaries, compaction, and message truncation
• tool requests, policy decisions, approvals, and outcomes
• file, credential, network, and connector access tied to the session

Alert on unusual session rewrites, history files changed by package installers, old sessions resumed from new devices, and tool calls that do not match the current user's request. A sudden jump from code assistance to reconnaissance, credential access, or external messaging deserves review even if the transcript claims earlier approval.

If poisoning is suspected, stop the agent before editing the evidence. Revoke its active credentials, isolate the workspace, preserve the original state files and operating-system metadata, and compare the loaded transcript with authoritative user and tool logs.

Do not simply delete the bad message and continue. Determine how write access was obtained, whether other sessions were changed, which tools acted after the modification, and whether any secrets or external systems were touched.

Incident Rule: Treat a forged approval as an identity event, not a content-quality bug. The attacker attempted to impersonate the authority that governs tool use.

A seven-point deployment checklist

Before allowing an AI agent to resume stored sessions and call consequential tools, verify the following:

1. History records are authenticated. Message authorship, order, and session identity can be verified.
2. Storage writes are restricted. Project code and ordinary dependencies cannot modify harness-owned state.
3. Approvals expire. Sensitive actions require current, target-specific authorization outside the transcript.
4. Tools are task-scoped. The agent receives only the capabilities and credentials needed now.
5. Policy is external. Network, data, and destructive actions face deterministic checks beyond model judgment.
6. Logs are reconstructable. Investigators can connect each loaded session to every tool request and outcome.
7. Tampering is rehearsed. A red-team test proves altered history is blocked, alerted, and preserved.

Do not wait for a vendor feature to begin. File permissions, isolated service accounts, short-lived credentials, outbound allowlists, and independent approval services can reduce risk now. Cryptographic record integrity should follow, but least privilege and policy enforcement already limit the blast radius.

The larger lesson for AI agent trust

Conversation history makes agents useful because it supplies continuity. It also creates a durable trust channel that attackers can target. A model cannot reliably distinguish a genuine prior exchange from a perfectly formatted forgery when both arrive as ordinary context.

The September 24 Darktrace disclosure gives defenders a concrete test: can someone who writes a session file manufacture permission for the agent's next action? If the answer is yes, the system has combined memory, identity, and authorization into one unverified text stream.

Break those functions apart. Authenticate history, evaluate authority at execution time, minimize tool access, and monitor the full chain from stored state to external effect. That turns conversation history from an invisible control plane into a governed security boundary.

Conversation history poisoning is not just another prompt trick. It is state tampering aimed at the agent's understanding of who asked for what, what already happened, and what it is allowed to do next. Secure that past before giving the agent power in the present.