Skip to main content
Search

# Agentic Inference

4 related articles

Subscribe to updates
  1. NVIDIA Blog

    NVIDIA's Vera Rubin debuts on MLPerf: up to 2x performance per GPU in training and over 10x throughput per watt for agentic inference

    NVIDIA's Vera Rubin platform entered MLPerf for the first time, with submitted results across training, post-training and agentic inference. In training, Vera Rubin NVL72 delivered up to 2x more performance per GPU than the highly optimized GB300 NVL72 in its debut; on the DeepSeek-V3 benchmark it scaled from 256 to 1,024 GPUs at about 97% efficiency, cutting training time from 16 minutes to about four. In its first MLPerf Endpoints submission, Vera Rubin NVL72 showed more than 10x the throughput per watt of GB300 NVL72 at the same response speed in a provisional Kimi K3 result.

  2. Vera Rubin NVL72's first agentic-serving results: up to about 5.7x GB300's throughput per megawatt

    AA-AgentPerf has published the first serving results for NVIDIA's Vera Rubin NVL72 on real coding-agent trajectories: on DeepSeek V4 Pro its total throughput per megawatt reaches up to about 5.7x the previous-generation GB300 NVL72, and about 5.4x, 3.0x and 2.8x on DeepSeek V4.1 Flash, GLM-5.2 and Kimi K3. Those multiples appear where GB300's throughput falls away as per-user speed rises, while at peak throughput the lead is only 1.14x to 1.45x. On average 97% of a request's input tokens repeat the previous request, making prefix caching and KV cache across memory tiers central.

  3. arXiv

    Agent serving stops restarting nodes to rebalance: Evolving Infra switches prefill and decode in place, cutting a switch from 754 seconds to 6.2

    For coding, math and computer-use agent workloads, Evolving Infra decouples compute allocation between prefill and decode from KV cache management: a node switches phase without a restart, and its KV stays readable from a shared pool. On up to 256 GB300 GPUs it cuts mean end-to-end latency and P95 time to first token by up to 4.1 and 14 relative to NVIDIA Dynamo, and a phase switch takes 6.2 seconds on average against 754 seconds for a Dynamo relaunch.

  4. arXiv

    JevSpawn lets agents derive finite action fields from natural-language tasks, topping five of eight benchmarks and cutting navigation latency to about 41 seconds

    JevSpawn introduces a training-free compositional policy that turns natural-language task specifications into executable finite action fields, exploring them through parallel action spawning, feedback-driven branch selection, representation revision, and recovery from retained branches; across eight benchmark tasks against seven agent baselines and a TypeSafe Jev variant it achieves the highest scores on five of eight tasks and, on Maze and Grid, raises success rates to 0.96 and 0.95 while reducing end-to-end latency from the fastest baselines' 47.91 and 61.44 seconds to 40.91 and 40.58 seconds.

You’re all caught up