Skip to main content
Search

Sofian Zalouk

1 related articles

Subscribe to updates
  1. arXiv

    Masked diffusion models learn to rewrite: LOOT revises generated tokens, beating ReMDM's 2048 evaluations with 256

    Once a masked diffusion model fills in a token it cannot change it, and training signal lands only on masked positions. The Leave-One-Out Transformer (LOOT) makes each position's prediction independent of that position's own input, so visible tokens can be supervised and generated tokens resampled during inference. On OpenWebText, LOOT with Blocked Gibbs correction reaches a MAUVE of 0.408 at 256 forward evaluations, above ReMDM's 0.348 at 2048; the cost is about 2.2 times the FLOPs per forward pass and a likelihood gap to standard bidirectional MDMs at longer sequences.

You’re all caught up