Where I started
A model can refuse a malicious request in a chat and still carry it out inside an agent. The request might arrive through a retrieved document, a tool response or a message from another agent.
For this survey, I read six studies across those interfaces. I wanted to understand what model-level safety tests miss once the model is part of a larger system. The findings below describe the studies reviewed in autumn 2025.
Three routes in
Prompt injection. InjecAgent tests instructions hidden in material an agent reads. WASP distinguishes between diverting an agent and getting it to complete an attack. That distinction matters: an agent may follow the wrong instructions but fail to finish the task. Poor execution can make a vulnerable system look safer than it is.
Poisoned retrieval and memory. AgentPoison places a small number of crafted examples into information the agent may later retrieve. The attack persists beyond one conversation. Fluent text can carry the trigger, so a filter looking for unusual language may miss it.
Messages from peers. Lupinacci and colleagues tested attacks relayed through other agents. In their experiments, models sometimes accepted requests from a peer that they had rejected from a human. The route a request took changed how it was treated.
The studies use different tasks, models and success criteria. Their attack-success rates shouldn’t be read as a single ranking of which architecture is safest.
The cost of another layer
In the original survey I called this the “abstraction tax”: each tool wrapper, memory store and orchestration layer adds another place where information can cross a trust boundary.
Wicaksono and colleagues found that the same assistant behaved differently depending on where an injection entered its workflow. Some attacks worked against the agent even when they failed against the standalone model.
Detecting a suspicious instruction is only useful if the system can stop it affecting the next action. A warning inside the conversation doesn’t itself prevent that text from becoming a tool argument.
Checking actions before execution
Progent was the most useful defensive example in the survey. It checks tool calls against explicit policies before execution, including which tools and parameters are allowed. Hand-written policies blocked all attacks in the reported benchmark runs.
Those results have limits. Policies still need to anticipate the relevant actions, generated policies allowed some attacks through, and restrictions could reduce success on legitimate tasks. The system also couldn’t express every constraint involving data passed between tools or user-specific trust.
The question I’m left with
If the model follows a malicious instruction, what can it actually do? I’d start there when assessing an agent: its permissions, execution environment, tool arguments and the information it can pass to other agents.
A useful evaluation needs to test those boundaries directly, alongside the model. Refusal tests alone don’t show whether an agent can keep an untrusted document from causing an unwanted action.
Written for UCL’s Networking and RTOS coursework. Studies discussed: InjecAgent, WASP, AgentPoison, Lupinacci et al., Wicaksono et al. and Progent.