Diffusion language models learn to rewrite their own tokens: PURL lifts GSM8K accuracy to 54.97% at matched unmask budgets
Masked diffusion language models write several tokens per step through parallel decoding, but tokens sampled independently within a step can contradict one another, and standard absorbing decoding never reopens a written token. PURL trains edit selection, unmask selection and token prediction jointly inside one normalized joint transition, so a single forward pass at inference supplies both unmasking and revision decisions. Across Sudoku, mathematical reasoning and code generation, accuracy improves by up to 47.4% at matched unmask budgets, and the TinyGSM model reaches 54.97% on GSM8K.