Agentic AI's OODA Loop Problem

TL;DR

Schneier and Raghavan reframe agentic LLM security through John Boyd’s OODA (Observe–Orient–Decide–Act) loop: classical OODA assumes trusted inputs and outputs, but agentic AI deliberately embeds untrusted actors at every stage — adversarial web data at Observe, poisoned pre-training and RAG corpora at Orient, fine-tuning / reward-hacking attacks at Decide, and tool / MCP confusion at Act. Each agent tool runs its own nested OODA loop, so attack surface multiplies with every MCP server and every sub-agent. The authors argue prompt injection is architectural, not a filter problem, and that the only durable fix is system-level integrity (input/processing/output verification) — analogous to TLS and Spectre mitigations — rather than better alignment. Published in IEEE Security & Privacy (Vol 23 Issue 3, DOI 10.1109/MSEC.2025.3604105).

Key claims

Boyd’s OODA loop — Observe, Orient, Decide, Act — has long been applied to AI/robotics; Anthropic’s working definition “agents are models using tools in a loop” is the same loop under a different name [§Introduction].
Classical OODA presumes trusted sensors, controlled environment, and physical boundaries; modern LLM agents violate all three by querying adversary-controlled sources mid-loop, ingesting poisoned RAG documents, and calling tool APIs that can execute untrusted code [§Untrusted Loops].
Fixing AI hallucination is insufficient: an agent that faithfully interprets a poisoned input and produces the corresponding output is still fully corrupt, so input-integrity is a strictly distinct problem from accuracy [§Untrusted Loops].
Prompt injection (Willison 2022) is architectural, not a filtering problem: a single token stream mixes untrusted data with trusted instructions, and there is no privilege separation between the data and control paths — the very mechanism that makes LLMs useful is what makes them vulnerable [§Prompt Injection].
Each tool an agent calls has its own OODA loop that nests, interleaves, and races with the parent loop; MCP-style protocols multiply attack surfaces because tool descriptions themselves become injection vectors and models verify tool syntax but not semantics [§Tools and MCP].
The four stages decompose into a risk taxonomy: Observe — adversarial examples, prompt injection, sensor spoofing; Orient — training-data poisoning, context manipulation, semantic backdoors; Decide — fine-tuning attacks, reward hacking, objective misalignment; Act — output manipulation, tool confusion, action hijacking [§OODA Risk Taxonomy].
Training-pipeline compromises persist through downstream improvement steps: fine-tuning preserves backdoors, RLHF adds human preferences without removing model biases, and each training phase compounds prior compromises — a Thompson-style “trusting trust” problem for ML [§Training-Pipeline Compromise].
Boyd’s “inside your adversary’s OODA loop” phrase inverts under agentic AI: adversaries are literally inside the loop because they supply the observations and can manipulate the actions; the competitive advantage (web-scale data) is identical to the attack surface, and a faster loop is worse because it shrinks the verification window — “millisecond decisions result in millisecond compromises” [§Inside the Loop].
The proposed defense direction is integrity at the system level (input integrity, processing integrity, output integrity) rather than better alignment, analogous to how TLS solved channel integrity and CFI/SMEP mitigated control-flow hijacking [§Toward Integrity].
Agent permissions, observability/auditability, and provenance tracking are listed as concrete near-term controls, but the authors do not claim they are sufficient — only that they bound the damage [§Toward Integrity].

Method

This is a position / framing essay, not an empirical paper. The “method” is to overlay Boyd’s OODA loop on the LLM-agent lifecycle and walk each of the four stages, naming a class of attacks at each one. The argument structure: (i) define OODA and its trust assumptions; (ii) show each assumption is violated by web-scale agentic LLM deployment; (iii) catalog the resulting attack classes; (iv) argue alignment can’t fix architectural problems; (v) call for integrity-by-construction. The essay synthesizes prior work (Willison’s prompt-injection framing, Thompson’s “trusting trust”, MCP security writeups, Microsoft’s EchoLeak CVE-2025-32711 as a worked example of agent-to-agent compromise via email) rather than proposing a new mechanism.

Results

No empirical results — the paper’s “results” are conceptual contributions:

A unified vocabulary (OODA stages) for what is currently fragmented LLM-security literature.
The “embedded untrusted actor” formulation: agents don’t just face adversarial inputs at the boundary, they invite the adversary into the loop because the adversary’s data is what makes the agent useful.
The “speed is a vulnerability” inversion of Boyd: agentic loops that beat humans on latency necessarily sacrifice per-step verification time.
The compounding-compromise argument: alignment-style mitigations (fine-tuning, RLHF, refusal training) cannot remove a backdoor seeded earlier in the pipeline, so any defense that operates post-training is structurally insufficient.

Why it’s interesting

This essay is the cleanest top-down framing on the wiki of why all the recent agentic-AI engineering work (research agents, MCP tool ecosystems, browser-using agents) sits on a security foundation that has not been built yet. It pairs sharply with Universal and Transferable Adversarial Attacks on Aligned Language Models, which gives the empirical “Decide-stage” datapoint Raghavan/Schneier need — universal/transferable GCG suffixes show that post-hoc alignment finetuning is fragile under cheap white-box optimization, exactly the “alignment can’t fix architectural problems” claim made here. It also extends Tool-Use Agents in an orthogonal direction: that concept page tracks how MiroThinker / InfTool / Training-Free GRPO / Kimi K2.5 scale the capability of agentic loops (interaction depth, parallelism, context-space optimization), while this essay asks what the threat model of those same loops looks like — every additional sub-agent and tool call in K2.5’s Agent Swarm is another nested OODA loop in Schneier’s taxonomy. Finally, the “fine-tuning preserves backdoors / RLHF compounds prior compromises” argument is the security-side restatement of the Training large language models on narrow tasks can lead to broad misalignment finding that narrow fine-tuning can induce broad misalignment via low-dimensional “persona” directions — both papers converge on the position that pipeline-internal interventions are not enough.