Skip to main content
Back to timeline
NVIDIA Technical BlogSource publication:

NVIDIA's Vera Rubin NVL72 tops 10x GB300 NVL72's throughput per kilowatt on agentic inference, with results still provisional

Synopsis

NVIDIA reports provisional MLPerf Endpoints v1.0 results in which Vera Rubin NVL72 delivers more than 10x the throughput per kilowatt of provisioned power of GB300 NVL72 on Kimi K3, at a shared minimum end-to-end interactivity of 50 tokens per second per user. The benchmark uses coding and enterprise workflows through API endpoints to weigh serving capacity against responsiveness, with separate SWE-bench Verified checks of end-to-end task solving. The Vera Rubin figures are pending MLCommons verification and may change under peer review.

Source-provided article image: NVIDIA Vera Rubin NVL72 Leads Agentic Inference Performance on MLPerf Endpoints v1.0
Figure 1 · NVIDIA 开发者技术博客

Interpretation

NVIDIA's provisional results put Vera Rubin NVL72 above ten times GB300 NVL72's throughput per kilowatt of provisioned power for agentic inference, while holding a floor of 50 tokens per second per user of end-to-end average interactivity. The comparison is against GB300 NVL72 results already verified by MLCommons, and it is drawn at a shared interactivity threshold rather than on raw total throughput. On Kimi K3, Vera Rubin NVL72 ran 32 accelerators at 1,024-query concurrency with system throughput of 41,414.2 tokens/s, end-to-end average interactivity of 50.4 tokens/s per user and operating power of 100.89 kW; GB300 NVL72 ran 48 accelerators at 48-query concurrency with 2,210.4 tokens/s, 56.2 tokens/s per user and 60 kW.

The result comes from co-design across hardware, system architecture and serving software, so the conversation history agents reuse is not recomputed each turn. Agentic workloads accumulate history across turns and mix reasoning, tool calls and responses, so a serving system must preserve reusable state, spread requests across workers and optimize prompt processing and token generation separately. Prefix caching and state reuse cut redundant computation, NVIDIA Dynamo handles routing and load balancing, and disaggregated serving lets prefill and decode be tuned independently; per-model kernels, fusion, memory access and speculative verification are tailored to each of the three models, such as DeepSeek's MegaMoE kernel using Rubin GPU locality and matrix-multiply instructions.

Perspective

What is shown is the submitted performance of three models, Kimi K3, DeepSeek-V4.1-Flash and Qwen3.6-35B-A3B, in the Standardized division, Available category of MLPerf Endpoints v1.0, compared as throughput per kilowatt at a stated minimum end-to-end average interactivity or as interactivity at matched concurrency. Readers should treat it as serving performance under a specific benchmark and hardware configuration, not as a general deployment conclusion.

The Vera Rubin submissions are provisional, pending verification by the MLCommons Association and subject to change under peer review; the GB300 results are verified by MLCommons. Throughput and interactivity comparisons select each platform's best point meeting the stated threshold, and the ratios are rounded.