Skip to main content
Back to timeline
arXivSource publication:

Diffusion language model Clover ships open weights: agent-benchmark lead, 680 tokens/s decoding on one GB300

Synopsis

DiffuSpace released Clover-35B-A3B-Preview, a mixture-of-experts diffusion language model with 35 billion total and 3 billion activated parameters that adapts Gated Delta Networks to block diffusion, keeping generated context in recurrent memory while tokens inside the current block interact bidirectionally. Under a shared evaluation protocol it leads seven agent metrics, scoring 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1, and reports 680 tokens/s on internal agent workloads with one NVIDIA GB300 GPU at BF16 and batch size one.

AI-generated editorial illustration: Clover-35B-A3B-Preview Technical Report

Interpretation

Diffusion language models can now compete directly on agent tasks: Clover leads all seven evaluated agent benchmarks, including 73.6 on SWE-bench Verified, 51.7 on Terminal-Bench 2.1 and 46.5 on SWE-Pro. Earlier public diffusion models trailed autoregressive baselines on agent tasks; the reproduced comparison shows LLaDA2.2 flash and DiffusionGemma below the evaluated Qwen baselines on both Terminal-Bench 2.1 and SWE-bench Verified. The 18 main-table benchmarks were reproduced under one evaluation protocol with each baseline's officially recommended decoding configuration; coding-agent tasks use Terminus 2 for Terminal-Bench and Codex for SWE-bench, with each task evaluated three times and averaged.

The model compresses completed context into a fixed-shape recurrent memory, while tokens in the current block read each other through bidirectional attention, and both kinds of information compete in one softmax. Gated Delta Networks are causal recurrences that summarize only what lies to a token's left, whereas block diffusion needs undecided tokens within a block to see one another. BD-GDN keeps the gated delta-rule recurrence on the clean path and, on the noisy path, reads the fixed prefix state at the block boundary and normalizes it together with block-local logits; history occupies a single softmax entry, so a learnable block-size calibration term compensates, and a zero-initialized right-context convolution branch adds 740,160 new parameters across 30 BD-GDN layers.

Decoding uses the same model's causal path to verify the draft block produced by one diffusion step, accepting a prefix according to an entropy threshold. Committing tokens by a fixed confidence threshold keeps draft quality and verification cost as separate concerns. On one NVIDIA H200 at BF16, batch size one and block size 16, Clover decodes at 465.2 tokens/s on MATH-500 versus 604.3 for Qwen3.6-35B-A3B with DFlash 2, but reaches 556.6 versus 396.7 on agent replay and 356.7 versus 286.5 on daily chat.

Perspective

Evaluation covers 18 benchmarks, seven of them agent tasks, with coding-agent tasks averaged over three runs; the speed figures come from single-H200 and single-GB300 settings at BF16 and batch size one. Weights, supervised fine-tuning code and the inference engine accompany the release, including dedicated infrastructure for running Clover locally on consumer devices.

The authors state that agent replay measures inference speed only, executes no commands and does not measure task success; the speed comparison involves different target models, verification trees and decoding policies, so the drafting algorithm's effect cannot be isolated. Difficult knowledge questions, competition mathematics and standalone code generation remain directions the authors list for improvement, and the internal decision sets were used repeatedly during development, serving as capability checks rather than new blind tests.