KAN symbolic extraction stops fitting edges one by one: end-to-end loss picks operators, cutting median test error by up to 99.8% on Feynman tasks
Synopsis
Kolmogorov–Arnold Network (KAN) edge functions can be replaced by symbolic operators to yield checkable analytical expressions, but the usual route fits each edge in isolation, making the choice sensitive to initialization and parameter fitting. GSR temporarily inserts a candidate operator, briefly fine-tunes the whole network, and commits the one with the best end-to-end loss; GMP instead learns operator preferences through differentiable gates during training, then discretizes with an optional short greedy refinement. On ten SRBench Feynman targets, one-factor sweeps put GSR or FastKAN+GSR at the lowest median test error on seven targets, and FastKAN+GSR lowers the median on Feynman I.12.1 from 9.49 to 0.0212.
Interpretation
KAN symbolic extraction now judges candidate operators in the context of the whole network rather than by how well a single edge curve fits. The earlier AutoSym route sampled each learned edge function separately and picked the library operator with the best local fit, so small changes in initialization, spline grid or training could yield different operators for the same dataset. On ten SRBench Feynman targets under one-factor sweeps over width, regularization strength and pruning cycles, GSR and FastKAN+GSR reached the lowest median test error on seven targets, while FastKAN+AutoSym was lowest on II.34.29a, II.6.15a and II.6.15b.
GSR converts edges one at a time by edge importance: it places a candidate operator on that edge, briefly retrains the full network, and uses validation loss to decide whether to commit. The selection borrows the greedy structure of matching pursuit, but the dictionary is the symbolic operator library and the improvement criterion is whole-network end-to-end loss, not residual norm or single-curve fit error. Experiments use a library of 25 univariate operator forms with 100 steps per candidate evaluation, a reference configuration of width 5, regularization coefficient 10^-2, 3 pruning cycles and seed 1, and 15 runs per dataset covering 12 distinct configurations.
GMP moves operator selection into training: each edge keeps differentiable gates over the operator library, entropy regularization and periodic top-k pruning push toward sparse choices, and after convergence the highest-probability operator is taken with an optional restricted greedy refinement. This avoids trying every candidate on every edge, amortizing the expensive in-context evaluation into training, at the cost that gate separation can lag behind greedy post-hoc conversion. Under the shared pruning cadence some GMP runs were unavailable at default settings or had fewer than three successful seeds; GMP had the lowest mean at that fixed setting on II.6.15a, which the authors flag for cautious interpretation.
Perspective
The conclusions are confined to ten targets from the SRBench Feynman with-units suite, a fixed training protocol and a library of 25 univariate operators, and measure test-error distributions under perturbations of width, regularization strength, pruning cycles and random seed rather than real scientific data or noisy, sparse, out-of-distribution regimes.
The authors note that one-factor sweeps cannot characterize higher-order interactions such as particular width–pruning combinations, and that affine reparametrizations make distinct operators hard to distinguish on a bounded domain, so correct recovery is not uniquely defined; some GMP runs are attributed to gates separating more slowly under the shared pruning cadence, a procedural failure rather than a definitive verdict.
