Agentic RL stops leaning on scalar rewards alone: targeted guidance teaches tool formats and writing styles that plain policy gradients could not
Multi-turn tool-calling agentic reinforcement learning is often limited by the base model's coverage, so some rewards cannot be learned even when the reward function is well designed. Targeted guidance places the textual feedback attached to each reward criterion just before the offending agent turn, builds an expert that shares the policy's weights but sees that feedback in context, and distills toward it only on that turn while combining it with a per-criterion RL reward. In a Qwen3-Coder-30B software-engineering multi-turn setting, this combination taught a new semantic-search tool argument format and a per-turn poem requirement while keeping coding pass rate near that of GRPO training alone.