Diffusion language model post-training routes by outcome: tool-calling beats the distilled checkpoint by 18.3 points, while math swings with the seed
Bidirectional denoising makes the policy likelihoods that PPO-style objectives need unreliable in diffusion language models, so a likelihood-free pipeline combines multi-teacher on-policy distillation, REINFORCE grouped by verifier outcome, and gradient-norm anchoring. On DiffusionGemma-26B-A4B at 24 denoising steps, the routed arm scores 18.3 points above the distilled checkpoint on tool-calling but 2.8 and 2.6 points below it on math at 24 and 48 steps; the anchored arm restores reasoning-trace accuracy, with a tool-calling difference whose confidence interval includes zero. A domain-conditional recipe keeps the tool-calling gain across two seeds, while the math result depends on the seed.