Skip to main content
Search

# Drifting

1 related articles

Subscribe to updates
  1. arXiv

    Seq-Drifting writes a whole sentence in one forward pass from noise, reaching 36.01 generative perplexity on LM1B

    Diffusion language models emit tokens in parallel but still need multiple sampling steps for high-quality text. Seq-Drifting moves drifting training onto sequences of continuous token representations, builds a context-dependent candidate set at each position, and forms a training target from attraction and repulsion, so the generator maps noise to a full sequence in one forward pass. On LM1B, Seq-DriftingP, which builds support sets with a public pretrained GPT-2, reaches a one-step generative perplexity of 36.01, below the 98.76 of the four-step distilled baseline FMLM.

You’re all caught up