Skip to main content
Back to timeline
arXivSource publication:

Safety harnesses stop being hand-rewritten: Evolution Under Fire evolves attacks and defenses together, cutting injection success to 3.0%

Synopsis

Tool-using language-model agents read emails, webpages and documents, so malicious instructions and legitimate task directives arrive through the same channel, while existing defenses are largely hand-designed and static. Evolution Under Fire couples evolution of complete executable safety harnesses with evolution of adaptive stressors in three stages: a utility-constrained Pareto tree search rewrites the harness, utility-preserving injections are generated, and those stressors feed back into further harness search. On AgentDyn with GPT-4o-mini as the task model, the evolved harness reduces attack success from 50.4% to 3.0% while raising clean and attacked utility.

Source-provided article image: Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors
arXiv

Interpretation

Agent defense shifts from picking one fixed mechanism to searching over a complete executable harness inside a fixed evaluation environment. Earlier prompt reminders, content filters, privilege policies, causal monitors and action auditors each occupy a fixed point in a large program space, and further improvement depends on repeated manual redesign. On AgentDyn with GPT-4o-mini as the task model, the selected harness reduces attack success from 50.4% to 3.0%, raises clean utility from 45.0% to 46.7% and attacked utility from 36.1% to 39.1%.

The search admits only candidates that keep task utility, so the security gain is not bought by disabling useful behavior. A candidate must retain at least 80% of the initial harness's successes on every required utility coordinate; candidates that miss the floor stay in the archive as counterexamples but cannot become parents. Among reproduced defenses in the same low-attack-success regime, none retains as much attacked-task utility.

The attack side evolves too, and a generated stressor is kept only when it breaks the target without destroying the original task. A fixed attack suite is an incomplete optimization signal, and static evaluations can overestimate defense efficacy once attacks are adapted to the target defense. Under the same search budget and starting state, stressor-guided evolution finds four utility-qualified harnesses with attack success at most 32/560, against one for the harness-only control; among candidates matching or exceeding the control on all three utility measures, the best attack success is 17/560 with stressor feedback and 146/560 without it.

Perspective

The demonstrated scope is the AgentDyn and Agent-SafetyBench benchmarks, with GPT-4o-mini and Qwen2.5-7B-Instruct as task models respectively, while the evaluation environment, tools, scorers and benchmark data stay fixed during evolution; cross-model transfer is tested only on AgentDyn with Qwen3.5-35B-A3B, with the harness frozen and no further evolution.

On Agent-SafetyBench with Qwen2.5-7B-Instruct as the task model, attack success falls from 82.6% to 62.0%, a smaller drop than on AgentDyn, and that benchmark releases no task-completion utility oracle, so helpfulness and runtime validity are added evaluation extensions. Stressor transfer across harnesses is also not universal: of 82 AgentDyn stressors that preserve utility, 43 achieve the adversarial objective against all four representative recipient harnesses and 36 also preserve utility on every recipient.