Quantized tensor kernels stop being hand-written per format: one algorithm and one schedule library generate 347 kernels, including 38 weight–activation pairs GGML lacks
Tensor libraries such as GGML hand-write a kernel for every weight format, activation format and instruction set; on the Arm CPU the authors tested, only 8 of the 54 weight–activation pairs GGML runs have a tiled matrix-multiplication kernel. The authors extend Halide's separation of algorithm and schedule to the representation of values: a codec describes a quantized format, and a new approximate_by directive turns its encoder and decoder into ordinary schedulable stages. One matrix-product algorithm and one schedule library generate 347 kernels covering 31 GGML and KleidiAI formats, including 38 weight–activation pairs for which GGML has no kernel.