Skip to main content
Back to timeline
arXivSource publication:

Long contexts may be unnecessary: agents that compact their working memory keep long-task accuracy while cloud token costs fall several-fold

Synopsis

On long-horizon and long-input tasks, frontier language models that repeatedly compact their conversation history into a short state, or store long documents as files and read them on demand, keep task accuracy broadly level with keeping the full history, with small gains or losses in individual settings. Because cloud serving charges by context length, shortening the context cuts estimated cost to a fraction. The authors sketch a semantic state space model (SSSM) that carries only a natural-language state between turns.

Source-provided article image: You don't need long context
arXiv

Interpretation

On agentic long-horizon tasks, compacting the conversation history into a very short context leaves frontier-model accuracy broadly unchanged while clearly lowering the cloud generation cost tied to context length. The usual setup appends every action and environment observation to one conversation that only grows, and longer contexts raise the latency, memory and price of each generated token. Across six benchmark families, SWE-bench Verified, SWE-bench Multilingual, Terminal-Bench 2.1, FRAMES, BFCLv3 and τ-bench, accuracy relative to full history ranges from -3.7 to +5.6 percentage points for Claude Opus 4.5 and from -3.3 to +4.4 points for GPT 5.6 Sol, with estimated cost savings of 2.2x to 8.8x and 2.4x to 5.0x respectively.

Long-input tasks do not need the whole document placed in context at once; storing documents in a filesystem and letting the agent retrieve them with command-line tools raises accuracy rather than lowering it. Earlier long-input baselines paste all documents into the model's working conversation, so peak context grows with document length. On LongBench v2, LOONG, AA-LCR, NIAH, RULER, Counting-Stars and BABILong, relative change ranges from -0.4 to +40.0 points for Opus 4.5 and from +1.0 to +30.9 points for GPT 5.6 Sol, with estimated cost savings of 6.5x to 54.6x and 4.4x to 233.3x.

Compaction can run after every single action, with almost no history carried between conversations, and task accuracy still stays close to the full-history baseline. Full-history accumulation is the default mode of the mini-swe-agent and mini-agent harnesses, where history is never removed or replaced. Under single-action compaction, Claude Opus 4.5 holds its baseline on SWE-bench Verified and gains 4.6 points on Terminal-Bench; GPT 5.6 Sol reaches 82.4% on SWE-bench Verified and 84.3% on Terminal-Bench (75 of 89 tasks solved), 2.0 and 3.3 points below its baseline.

Perspective

The conclusion rests on agentic workflows: six long-horizon and seven long-input task families, with principal experiments on Claude Opus 4.5 and GPT 5.6 Sol and supplementary runs of five quantized open-weight models on SWE-bench Verified and Terminal-Bench 2.1. The cost figures are estimates derived from throughput measurements and retail prices, not measured bills.

The authors state that the evidence does not rule out tasks where long context is beneficial, nor prove that every task can be solved with short context. Compaction is lossy, and constructing a task that must lose information to compaction is not difficult; among open-weight models, Qwen 3 Coder Next falls sharply under tight budgets, showing that robustness to compaction varies by model.