Agents learn online without only reading forward: revising retrieved reflections from trajectories beats memory and prompt optimization on three benchmarks
Synopsis
Agents running over long interaction streams must adapt online to distribution shifts, yet existing agent harnesses either evolve their state through forward information flow alone or rely on extra replay or validation data for optimization. ReHarness combines forward read and write with backward revision under a strictly online streaming protocol: after each instance, the resulting trajectory becomes feedback that revises the reflections retrieved for that instance, propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, it consistently outperforms raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches.
Interpretation
An agent harness gains a backward path for state updates: after each instance, the trajectory just produced revises the reflections retrieved for that instance, so feedback travels backward through the textual state. Existing agent harnesses typically evolve state only through forward information flow, or need additional replay and validation data to optimize the harness. Across three continual learning benchmarks and three language model backbones, the approach consistently outperforms raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches.
Revisions land on the textual harness state rather than model weights, which keeps the whole procedure inside a strictly online streaming protocol. The question it asks is what information could have been captured differently in the harness state to better inform the current instance in the stream. Analysis reports that the revisions are source-grounded and meaningful, correct overgeneralized or outdated experience under distribution shifts, and improve both stability and plasticity during continual learning.
Perspective
The work targets agents operating over long interaction streams, in the setting of continual learning under a strictly online streaming protocol, evaluated on three continual learning benchmarks and three language model backbones. Readers concerned with offline batch training, or with settings that supply extra replay and validation data, need to judge applicability separately.
The summary gives no concrete metric values per benchmark, no quantification of revision quality, and no indication of how large the differences are across backbones. Whether the gain from backward revision changes with stream length or shift intensity is not stated.
