A 114B audio-video model lands on Moore Threads: MAGI-2 Preview cuts a denoising step from 15.6 s to 565 ms
Synopsis
Sand AI's MAGI-2 Preview, an audio-video generation model with about 114B parameters and about 6B active per token, previously ran only in the CUDA ecosystem. With the compatibility layer torchada, vLLM-Omni's existing implementation runs almost unchanged on Moore Threads MUSA, and step-by-step optimization along the execution path took the first working version from 15.6 s per denoising step to a newest measurement of 565 ms. Those figures come from different code trees and sessions and are compared only within their own sections.
Interpretation
A 114B audio-video generation model written for CUDA now runs on Moore Threads MUSA and has been optimized along its execution path to hundreds of milliseconds per step. The first working version needed 15.6 s per denoising step in a 4-step diagnostic run, and most of that time was not compute but host synchronization and waiting on communication. On 8 MTT S5000 GPUs at 272p, 100 steps and static compilation, with every change applied, the newest measurement is 565 ms per step; that figure and the 15.6 s come from different trees and sessions and describe the work as a whole, not two code states against each other.
The reuse comes from a compatibility layer: CUDA-style PyTorch calls, the Inductor compilation path and distributed communication carry over directly, and the Triton kernels change only in per-platform launch configuration. torchada maps torch.cuda, cuda device strings, NCCL to MCCL and CUDA Graph to MUSAGraph onto torch_musa at import time, so the model code needs almost no rewriting. On 8 MTT S5000 GPUs, the upstream fused BF16 MoE, SwiGLU7 and mHC Triton kernels run as-is, with only launch parameters such as BLOCK_SIZE_K set to MUSA values.
Most of the per-step drop comes from removing host synchronizations and replacing a per-expert Python loop with fused kernels. Replacing the per-expert .item() with a single tolist() took SP8 from 12.2 s to 8.8 s per step, and replacing the per-expert loop with MATE's grouped GEMM took a single projection from 21.9 ms to 0.69 ms. On 8 MTT S5000 GPUs, a fused W13 plus SwiGLU7 Triton kernel took per-step time below 1 s for the first time, from 1464 ms to 954 ms, against a baseline from a 20-step run.
The last round splits the W13 gate and up accumulators into two half-width tiles so the SwiGLU7 epilogue runs on registers. The shipped kernel accumulated the interleaved gate and up rows in one FP32 shared-memory tile and read the epilogue from that tile, which limited how many programs stayed resident on an MP. Alternating control and candidate within one session on 8 MTT S5000 GPUs at SP8×CFG1+EP4, 272p, 100 steps and static compilation, per-step time fell from 690.8 ms to 590.4 ms, a 14.5% reduction, with all four runs sharing one output hash.
Perspective
The demonstrated scope is 272p text-to-audio-video and image-to-audio-video on 8 MTT S5000 GPUs, where image-to-audio-video has only generated samples and has not been benchmarked; 540p was measured only outside the timed sessions, and the 1080p refiner stage exists as draft #8915, which 272p does not execute.
Different sections use different code trees and sessions, so absolute numbers are compared only within one section and no delta is claimed between 565 ms and 670 ms; the expert-parallel-flag-off and EP8 configurations were each measured once, so only EP4 has run-to-run spread evidence; main with only merged PRs has not been fully measured on MUSA.
