Skip to main content
Back to timeline
NVIDIA BlogSource publication:

Lambda links over 10,000 GPUs with co-packaged optics: rack-scale network latency falls about 4%, optics run at 36°C median

Synopsis

Lambda has deployed NVIDIA Photonics co-packaged optics (CPO) switches in an AI factory, packaging optical engines alongside the switch chip to connect NVIDIA GB300 NVL72 racks carrying more than 10,000 GPUs. Company tests show network latency down about 4% at rack scale and up to about 10% at pod scale, with throughput matching conventional pluggable optics. Against pluggable modules, the switches are claimed to deliver up to 5x better power efficiency and 15x higher reliability, use 4x fewer lasers, and run optics at a 36°C median versus 51°C.

Source-provided article image: Lambda Boosts AI Factory Resiliency and Efficiency With NVIDIA Photonics
NVIDIA Research

Interpretation

The way an AI factory wires its GPUs has changed at scale: Lambda uses NVIDIA Photonics co-packaged optics switches to connect NVIDIA GB300 NVL72 racks spanning more than 10,000 GPUs, aiming at lower power, higher resiliency and longer mean time between interruptions. AI cluster networks previously relied on pluggable optical transceivers, with electrical-to-optical conversion outside the switch package and a failed laser requiring replacement of the whole module. Lambda's tests show network latency down about 4% at rack scale and up to about 10% at pod scale, while throughput stays the same as conventional pluggable optics across rack, scalable unit and pod configurations.

The power and reliability gains come from moving electrical-to-optical conversion next to the switch chip: optical engines share the package, the electrical path shrinks from inches to millimeters, and retimer chips are no longer needed. NVIDIA Photonics replaces pluggable transceivers with co-packaged optical engines on the switch package, claimed to deliver up to 5x better power efficiency and up to 15x higher reliability while using 4x fewer lasers. Each switch in Lambda's deployment has 144 ports running at 800Gb/s, liquid cooling carries heat away from the switch chip and optical engines, and the company reports a 36°C median optics temperature versus 51°C for conventional pluggable optics, with tests finding more stable signal integrity.

Serviceability is built in: each switch carries 18 removable external light sources that supply light to the optical engines, and technicians can replace those light-source modules in the field. Traditional pluggable optics require replacing the entire module when a laser fails, whereas here only the light-source module is swapped. This field-replaceable capability is part of the NVIDIA Photonics switch configuration that appears in Lambda's deployment alongside the 144 800Gb/s ports and liquid cooling.

Perspective

The report covers Lambda's deployment of NVIDIA Photonics co-packaged optics switches connecting NVIDIA GB300 NVL72 racks across more than 10,000 GPUs, together with the company's latency, throughput, temperature and reliability figures; these numbers describe that deployment and vendor testing, not a cross-vendor general conclusion.

The roughly 4% and up to roughly 10% latency reductions, the 5x power-efficiency and 15x reliability claims, and the 36°C versus 51°C temperature comparison are company reports or test results without independent third-party verification in the material; the improvement in mean time between interruptions is also given without a specific figure.