MIAO narrows accumulators row by row: MX systolic arrays save up to 38.7% area with 1.06% extra perplexity
MIAO replaces uniform floating-point accumulation across an MX-quantized systolic array with per-row accumulator precision and jointly selects product precision, searching for the lowest area and energy configuration under an end-to-end perplexity constraint. Relative to an all-BF16 intermediate-arithmetic baseline, it cuts processing-element array area by up to 38.7% and energy per operation by up to 36.1% across MXFP8, MXFP6 and MXFP4, with at most 1.06% additional perplexity degradation over operand-only MX quantization. Configurations selected on TinyLlama transfer to five further models without retuning, adding at most 1.6% perplexity.