Skip to main content
Search

# NVIDIA Cosmos 3 Nano

1 related articles

Subscribe to updates
  1. NVIDIA brings TensorRT multi-device inference into Dynamo-Triton 26.07 behind a single gRPC endpoint, cutting Cosmos 3 Nano video generation end-to-end latency from 156.595 s to 34.183 s

    NVIDIA describes integrating TensorRT multi-device inference (fully supported starting with TensorRT 11.0) into Dynamo-Triton 26.07, where one KIND_MODEL instance can own multiple GPUs, create per-rank TensorRT execution contexts, CUDA streams, and NCCL communicators, and launch the ranks together, so an application calls one named model through one gRPC endpoint; in a Cosmos 3 Nano video generation example, a 36-layer denoising transformer uses Ulysses context parallelism to distribute 44,160 video tokens across as many as eight GPUs, dropping end-to-end latency from 156.595 s on one GPU to 34.183 s on eight (4.58x), with 6.09x transformer RPC speedup and CP2/CP4/CP8 all passing the configured MAE ≤ 25 and PSNR ≥ 18 dB thresholds.

You’re all caught up