Skip to main content
Back to timeline
NVIDIA Technical BlogSource publication:

cuBLAS moves double-precision multiplication onto integer Tensor Cores: DGEMM up to 5.5x faster, CCSD 2x end to end

Synopsis

NVIDIA's cuBLAS in CUDA Toolkit 13.4.2 adds two algorithms, Ozaki-II and 2M, that split 64-bit floating-point matrix multiplication into integer pieces, multiply them on Tensor Cores and reassemble the exact result, speeding up DGEMM and ZGEMM with no application code changes. On VR NVL72 and GB200, Ozaki-II DGEMM runs 5.5x and 3.9x faster than native FP64, and a quantum-chemistry CCSD case is 2x and 1.4x faster end to end, with full-precision results indistinguishable from native FP64.

Interpretation

In CUDA Toolkit 13.4.2, cuBLAS no longer forces double-precision dense matrix multiplication to trade accuracy for speed: DGEMM and ZGEMM automatically take an integer Tensor Core path, and application code needs no changes. Previously these operations used the GPU's native FP64 units directly, and rapid advances in lower-precision hardware motivated exploring algorithms that deliver full double-precision results on the faster integer Tensor Cores. On VR NVL72 and GB200, cuBLAS DGEMM with Ozaki-II runs 5.5x and 3.9x faster than native FP64, and 2x faster than Ozaki-I on GB200.

The method aligns exponents, converts each double-precision number into a shared-exponent variable-length integer, accumulates products exactly on integer Tensor Cores, and reassembles the original result. Ozaki-II instead uses a big-integer multiplication based on the Chinese Remainder Theorem: each quantized operand is reduced modulo a set of small pairwise-coprime integers, one integer GEMM is performed per modulus, and the exact product is reconstructed from the residues. At the accuracy required to match FP64, Ozaki-I needs roughly twice as many Tensor Core GEMMs as Ozaki-II, because Ozaki-I's GEMM count grows with the square of the slice count while Ozaki-II's grows linearly with the number of moduli.

Complex matrix multiplication is accelerated too: the 2M algorithm cuts a complex product from four real GEMMs to two, so a ZGEMM costs roughly twice a real GEMM of the same size instead of four times. The conventional route computes a complex product as four real GEMMs of the same size; the 3M algorithm reduces this to three but, in floating-point arithmetic, forming the sums and subtracting intermediate products can degrade numerical stability. With exact integer operations the additions, subtractions and products introduce no error; on GB200 ZGEMM with 2M runs 5x faster than native complex FP64, and the BerkeleyGW case is 6.6x and 3.9x faster on VR NVL72 and GB200 respectively.

Perspective

The demonstrated scope is cuBLAS in CUDA Toolkit 13.4.2 for DGEMM and ZGEMM on VR NVL72, GB200 and RTX PRO 6000, plus two application cases: quantum-chemistry CCSD and materials-science BerkeleyGW. The CCSD case is frozen-core CCSD/cc-pVTZ on five mid-sized members of the S22 database containing 24 to 28 atoms and roughly 590 to 630 atomic orbitals.

The upper bound of the performance gain depends on how much integer Tensor Core throughput a GPU has relative to its native FP64 units, and that ratio varies widely across the product line; at the smallest matrix shapes ADP determines that Ozaki-II offers no advantage and dispatches to native FP64. The 32-mantissa-bit fixed-precision mode gives energy errors below 10^-10 Eh, sufficient for the vast majority of practical applications but not full precision; in full-precision mode CCSD correlation-energy errors are below 10^-14 Eh. The BerkeleyGW accuracy comparison found Epsilon matrix elements agreeing to 10 decimal places.