AutoScaffold writes temporary training hints, then removes them: students still beat RL baselines by 7.2 points on ALFWorld
Reinforcement learning scores each attempt with a single outcome reward, so the reasoning, actions and feedback inside a rollout are largely discarded, and information in failed trajectories is wasted. AutoScaffold has a frozen teacher model periodically read the student's training log, diagnose bottlenecks, and write class-level strategies plus hints for individual hard tasks, with class-level edits checked on held-out tasks before deployment. Evaluated without scaffolds, students beat the corresponding RL baselines in all four domains, by 7.2 points in ALFWorld success and 4.0 points in average QA exact match.