NVIDIA turns physical-AI training data into a pipeline: two prompts run five blueprints, and defect dataset assembly drops from two months to one day
Synopsis
NVIDIA released Cosmos Data Factory, packaging curation, labeling, augmentation and synthetic data generation into five cloud blueprints and six composable skills that frontier AI agents orchestrate from prompts, aiming to turn hard-to-collect training data into a controllable compute workload. On a server with eight RTX PRO 6000 Blackwell GPUs, the team converted daily throughput into cost: about 130 autonomous-driving edge-case clips per day at $2.66 in GPU compute per clip, and defect dataset assembly cut from two months to one day. These figures cover compute only; source data preparation, setup, agent use, storage and human review may add cost.
Interpretation
Physical-AI training-data preparation moves from disconnected tools and manual steps to a prompt-driven cloud workflow: two prompts let a developer run environment preflight, setup and a blueprint, covering autonomous-driving scenario generation, image attribute augmentation, video augmentation, rare-event video generation and defect image generation. The prior condition was curation, labeling, generation and evaluation done with disconnected tools and manual processes, while rare defects and nighttime incidents are themselves costly and slow to capture. All five reference blueprints run on the same CDF Orchestration core skill, which coordinates workflow steps, deploys required services and manages execution on the cluster; the first prompt runs preflight, which checks the environment without making changes and flags anything missing, before setup prepares it.
Each task chains retrieval, generation and evaluation: in the autonomous-driving blueprint, Cosmos Curator extracts video metadata, Cosmos Dataset Search finds relevant videos through embeddings, Cosmos3-Super-Image2Video applies style transfer, and Cosmos Evaluator checks the generated videos for physical plausibility. Retrieval combines metadata, scene semantics, spatial relationships and temporal behavior through multi-embedding retrieval; Cosmos Dataset Search is now open source under Apache 2.0, with adaptive agentic reranking in early access. The defect image blueprint ships fine-tuned for printed circuit boards, glass and metal, starting from clean inspection images or 3D USD assets, rendering inspection images and masks with Isaac Sim, creating clean golden images with Qwen-Image-Edit, and having Cosmos 3 Super generate defects from masks and requested defect types while preserving the surrounding part.
Cost is derived from daily throughput: on a cloud server with eight RTX PRO 6000 Blackwell Series GPUs at roughly $40 per server-hour, or $960 per day, image attribute augmentation produces about 42,649 annotated image pairs per day at $0.023 per pair, video augmentation about 305 annotated clips per day at $3.15 per roughly 5-minute clip, and event video generation about 384 annotated clips per day at $2.50 each and about 4 minutes per clip. The comparison is third-party purchased data and on-road collection: $0.25 average per image pair, $22 average per video, and $125 per scenario for a fleet crew. On that basis, image attribute augmentation is about 10x cheaper than third-party purchase, video augmentation about 7x cheaper, and autonomous-driving edge cases at $2.66 per clip about 45x cheaper than fleet collection; defect image generation cut dataset assembly from two months to one day.
Perspective
The report covers NVIDIA's published Cosmos Data Factory blueprints, skills and throughput-based cost conversion, for developers building physical-AI training datasets; the stated cost covers GPU compute only, and source data preparation, setup, agent use, storage and human review may add cost.
The cost figures are the team's conversion of daily throughput on a cloud server with eight RTX PRO 6000 Blackwell Series GPUs, not a general conclusion across hardware configurations; additional lidar devices, radar and other sensor types are planned expansion, and adaptive agentic reranking is in early access.
