Interpretable regression moves past pairwise interactions: MJBoost draws third-order and higher structure with Kolmogorov–Arnold units
Synopsis
Interpretable tabular regressors usually keep only main effects and pairwise interactions, so accuracy drops when a target depends jointly on three or more inputs. MJBoost replaces each boosting round's weak learner with a Kolmogorov–Arnold unit whose inner and outer functions are penalized splines, letting the model represent interactions of any order and admit an exact functional ANOVA decomposition. On synthetic targets of interaction order three to six its test error is up to 13 times lower than EBM's, and its variance shares match the true shares to within 0.04; on the 35 OpenML-CTR23 regression tasks it beats EBM on 21 and is worse on 2.
Interpretation
Interpretable tabular regression can now include joint effects of three or more inputs instead of falling back on a black box with post hoc explanations. Earlier interpretable models, including the explainable boosting machine EBM and its neural variants, restrict predictions to main effects and pairwise interactions; when a target depends on three or more inputs, they keep only the best main-plus-pair projection and lose the remainder. On synthetic targets of interaction order three to six, MJBoost's test error is up to 13 times lower than EBM's; on the exponential family at interaction order three, normalized test RMSE is 0.017 for MJBoost, 0.185 for its order-2 truncation and 0.206 for EBM, so the accuracy comes from units of order three and higher.
Each boosting round adds one Kolmogorov–Arnold unit that touches each input only through a single inner curve, so the fitted model splits exactly into main effects, pair effects and a higher-order remainder. The unit's inner and outer functions are penalized B-splines, fitted in closed form by alternating penalized least squares rather than stochastic gradient descent, starting from principal Hessian directions of the residual. On the 35 OpenML-CTR23 regression tasks, main and pair effects together explain a median 0.992 of the fitted model's variance; dropping the higher-order part raises test RMSE by a median 4.1%, and by 1981%, 1166%, 401% and 252% on the FPS benchmark, video transcoding, auction verification and naval propulsion plant tasks.
Perspective
The demonstrated scope is synthetic interaction targets, the 35 regression tasks of OpenML-CTR23, and the 32 binary and 11 multiclass tasks of OpenML-CC18, all fitted on one CPU thread; the code is released as the Python package mjboost with a scikit-learn interface and the functional ANOVA decomposition.
The higher-order part is small on most tasks, with main and pair effects explaining a median 0.992 of the variance, so the readability gain depends on the data; the full CTR23 run used 130 CPU-hours, about 1.91 times EBM, and the advantage over EBM on classification tasks is not general.
