Skip to main content
Search

# Prefix-Block Policy Optimization

1 related articles

Subscribe to updates
  1. arXiv

    Diffusion language models get block-level RL: PBPO clips per token and holds its own across six benchmarks

    Diffusion large language models decode block by block, denoising several tokens within each block, and the exact likelihood of a completed block is intractable because it marginalizes over intermediate states and reveal orders, so block-level policy optimization has lacked a usable objective. PBPO builds a prefix-block state and uses a single model evaluation to define a factorized one-step surrogate for the block likelihood, whose block surrogate ratio is the product of its token surrogate ratios, then applies surrogate importance weighting and clipping separately to each token and resamples corrupted views of the prompt and revealed prefix. Across two LLaDA-family initializations, six benchmarks and three generation lengths, PBPO performs competitively against existing dLLM RL methods.

You’re all caught up