Skip to main content
Back to timeline
PyTorchSource publication:

Helion stops autotuning to pick kernel settings: compile-time heuristics reach 1.18x of hand-written Triton on linear attention

Synopsis

Helion, PyTorch's performance-portable kernel language, now reads a kernel's intermediate representation and dataflow at compile time, sorts it into matmul, reduction or pointwise, and emits block sizes, warp counts and pipeline stages directly instead of running a slow autotuner. The previous no-autotune default ran at 0.52x of hand-written Flash Linear Attention Triton on H100, while autotuning reached 1.26x at the cost of GPU hours; the heuristics reach 1.18x across nine linear-attention variants and six production shapes, adding only tens of milliseconds of compile time.

Source-provided article image: Fast by Default: Compile-Time Heuristics for Helion Kernels
PyTorch Blog

Interpretation

A Helion kernel can now get a near-autotuned configuration without running the autotuner at all, moving the language's default behavior from behind hand-written implementations to ahead of them. Previously there were two options: the no-autotune default, at 0.52x of hand-written Flash Linear Attention Triton on H100, or GPU-hours of autotuning to reach 1.26x. Across nine linear-attention variants and six production shapes (batch 1 to 8, 8 to 96 heads, sequence 1024 to 16384, head dimension 64 to 256), the compile-time config reaches 1.18x of hand-written FLA Triton in the forward pass on H100 and 1.12x with backward included, and is 2.13x faster than Helion's previous default.

The same heuristics hold on production kernels written by someone else, so they are not tied to one specialist library. vLLM's six reduction kernels already had Helion implementations; Helion's no-autotune default ran at 0.58x of vLLM's hand-written CUDA/C++, and full autotuning reached 1.31x. Over 153 kernel-and-shape cells across six production kernels on H100, the computed config reaches 1.27x of vLLM's hand-written CUDA/C++, 2.2x faster than the previous default and about 3% short of full autotuning.

The configuration comes from compile-time analysis of kernel structure rather than from compiling and benchmarking thousands of candidates. One Helion kernel can lower into thousands of Triton kernels, and the autotuner must compile and time them to choose, a cost that multiplies per kernel, shape and hardware backend. The heuristic scans the device IR for operations: any kernel containing a matmul such as hl.dot or torch.baddbmm routes to the matmul heuristic, a reduction without matmul to the reduction heuristic, and everything else to the pointwise heuristic; if extents are unknown at compile time no heuristic fires and Helion falls back to the original default.

Perspective

The results cover the Triton backend on H100 and B200, where the heuristics are merged into Helion and enabled by default; the evaluated objects are linear-attention variants, six vLLM reduction kernels, and general reduction and matmul kernel suites.

The heuristics predict what another compiler will do with registers, shared memory and tensor memory, and that estimate is approximate; when some kernel extents are unknown at compile time no heuristic fires and Helion falls back to the old default; on jagged HSTU attention the old default happens to land on a good config (3.72x) and the heuristic does worse (3.27x); the matmul comparisons are not always apples-to-apples, since some PyTorch implementations materialize intermediates that the Helion kernel fuses.