CompoundingKernel searches GPU kernels from zero and tops all three SOL-ExecBench boards, 4.40x faster than the PyTorch baseline on 209 problems
Synopsis
CompoundingKernel turns GPU kernel writing into a search that starts from zero: each problem gets only its PyTorch definition, with no reference kernels and no official scores, while an archive outside the model keeps every correct kernel and its lineage and an LLM controller picks a parent or a new design and gives each parallel child agent one direction. As of 2026-11-11 its kernels rank #1 on the L1, L2 and Quantization public boards of SOL-ExecBench, all 235 problems have a site-verified correct kernel, and on the 209 public-board problems the kernels are 4.40x faster than the benchmark's PyTorch baseline.
Interpretation
GPU kernel writing is organized as a search that compounds rather than a one-shot code generation: each problem starts cold from its PyTorch definition, and correct kernels produced along the way stay in an archive outside the model with their timings and parent links, so later attempts build on those records. Most strong recent kernel results come from a fixed frontier model inside a search or harness, and a goal-only agent is already a strong baseline, so a harness has to earn its place by measurement. At equal wall time, the LLM controller beat a coded tree search that ranks nodes with a UCB-style score on 35 of 38 paired problems; per attempt the two picked about equally well, and the controller won because it made 2.4 times as many correct attempts.
Each search step is decided by a controller that reads the archive, chooses an existing kernel to refine or opens a new design, and hands one direction to a free child agent. A child gets only a parent and one direction, edits, compiles and times the code on a B200, and correctness is decided by checks outside the model, with only correct kernels written to the archive as nodes. Over a thousand agent sessions ran at once in the final run, and 98.4% of the candidates they evaluated were correct as of 2026-11-02; one warm restart fix raised their evaluation rate 3.6 times.
The limit on scale is the rate of reliable measurements, not the number of agents: the search compounds only when each new try builds on a reliable measurement of the last one. More guidance did not help, since making the optional references, a skill and a public wiki, mandatory added nothing measurable. The best path of Quantization problem No.030 rose from 6.25 to 27.36 on local timing along its lineage, yet 76 of its 270 steps were level or slower, showing that slower kernels can serve as stepping stones.
Perspective
The demonstrated scope is the 235 SOL-ExecBench problems on NVIDIA B200: kernels for the L1, L2 and Quantization public boards were scored and timed by the site, while FlashInfer-Bench kernels were verified correct on the site as private rows and their results are not published. Ranks, per-problem counts and speedups are as of 2026-11-11, and line counts and source patterns as of 2026-10-31.
The top ranks are not stable: L1 has been held since 2026-10-27 apart from a short loss on 2026-11-07, and other teams moved ahead on L2 on 2026-11-07 and on Quantization on 2026-10-27 and 2026-11-08; the public scores did not drop, and #1 was retaken when more rows were made public late on 2026-11-10 (UTC). Local times read fast because the GPUs run the SM clock unlocked at up to about 1,965 MHz, and people correct for this with a calibration the agents never see.
