Skip to main content
Back to timeline
arXivSource publication:

LLM agents learn environment facts and execution tips on separate tracks: TTSE beats a unified rule bank by about 3 points across five SOPBench domains

Synopsis

Huawei's Poisson Lab introduces TTSE, which splits what an LLM agent learns from interaction into two independently evolving tracks: facts about what the environment is like, and procedures for what to do under which condition. Prior self-evolution work treated environmental knowledge as fixed external input or optimized skills alone. Across 892 tasks in five SOPBench domains and three repeats, the dual-track version scored 2.99 percentage points above a unified single-track bank, with a 95% confidence interval of 2.34 to 3.65 points.

Interpretation

Experience an agent accumulates from interaction is kept in two separately maintained classes: fact rules describing regularities of the environment, and procedure rules carrying an explicit condition prefix, each with its own numbering, injection, verification and retirement. Earlier self-evolution work, whether reflective feedback, skill libraries or rule extraction, centered on what behavior to adopt for a task, while environmental knowledge arrived as static external input such as task descriptions or prompt context rather than an object that could be accumulated, verified and revised. In the four-condition GDPevo ablation, the full dual-track version averaged 0.2616 over three runs, procedures-only 0.2380, facts-only 0.1918 and the zero-shot baseline 0.0947; the full version leads procedures-only by 2.4 points and facts-only by 7.0 points.

The two tracks update in a chunked online loop, and each chunk ends with blame, retire, contradiction synthesis and induction, so rules learned wrongly early can be removed in later chunks. An offline pipeline collects all trajectories before a one-shot induction and does not self-correct its rule library during training; TTSE instead triggers one library update per completed chunk. In the blame phase each failed trajectory in the current chunk is checked for a single injected rule that directly misled the agent, with at most one rule blamed per failure; blamed rules move to a retired pool with the retirement reason kept for later contradiction synthesis.

Combined with Bayesian posterior-driven skill management, the dual-track representation still adds accuracy, so the split does not depend on TTSE's own count-based evidence tracking. The single-track ablation puts all rules into one unified bank without a fact/procedure distinction and lets only Bayesian posteriors govern rule quality, so the two configurations differ only in track structure. On 892 tasks across five SOPBench domains with three repeats, single-track reached 91.59% overall and dual-track 94.58%, with a paired bootstrap 95% confidence interval of 2.34 to 3.65 points and a permutation p below 10 to the minus 4; excluding the warehouse domain that supplies most of the gain, the advantage stays positive at 0.67 points with p of 0.011.

Perspective

The demonstrated scope is evaluation on five benchmarks: ALFWorld, ScienceWorld, GDPevo, five SOPBench domains and 147 end-to-end PinchBench tasks, all reported as means over three independent repeats; PinchBench uses a streaming protocol with no held-out test set and guarantees only that a task's own solution is never injected back into that same task.

The dual-track advantage varies by domain: dangerous goods scores 94.0% dual versus 94.6% single, a non-significant difference, and the warehouse domain supplies 78% of the total gain, leaving 0.67 points once it is excluded. The gap between the full version and procedures-only carries no formal significance test, and the authors frame it as ablation evidence rather than a statistically certified margin. The rule bank starts from a blank cold start, so the initial chunk relies entirely on the model's zero-shot ability, and the induction stage lacks structured execution verification.