Quadratic manifold fitting drops the fixed noise assumption: AMLE-SQMF estimates a shape parameter from data and matches fixed-exponent fitting on face reconstruction
Synopsis
High-dimensional data often lie near a low-dimensional curved manifold, and quadratic manifold fitting approximates that structure with local quadratic surfaces, but earlier methods treat residuals as Gaussian and fix the loss. AMLE-SQMF instead models residuals with a generalized Gaussian distribution whose shape parameter is unknown, estimating that shape parameter jointly with the manifold parameters by maximum likelihood so the fitting loss adapts to the data. On synthetic spheres and noisy face images the method recovers the noise shape parameter, its face reconstruction error is lower than the compared manifold-denoising methods in most tested conditions, and against fixed-exponent fitting it keeps competitive accuracy.
Interpretation
The residual model in quadratic manifold fitting is no longer chosen in advance by the user; a shape parameter that controls tail thickness is estimated from the data, and the fitting loss changes with it. Earlier quadratic fitting methods mostly use a squared Euclidean loss, which corresponds to a Gaussian noise assumption and is less robust to heavy-tailed residuals and outliers; residuals also contain approximation error from the second-order truncation, so their distribution need not be Gaussian. With 1000 points sampled uniformly from the unit sphere and independent generalized Gaussian noise added, and with the fitting exponent matched to the noise shape, optimization time is shorter than standard gradient descent in all four exponent settings compared, with the largest reductions at exponents 1.25 and 1.75.
The approach writes residuals as a generalized Gaussian distribution, which reduces to Gaussian at shape parameter 2 and to Laplace at shape parameter 1, then estimates manifold parameters and this shape parameter together by maximum likelihood. The scale parameter is removed through its closed-form maximum likelihood estimate, leaving a profile negative log-likelihood over only the manifold parameters and the shape parameter and avoiding direct optimization of the scale variable. Optimization alternates between two steps: with the shape parameter fixed, gradient descent fits the quadratic manifold, and the orthonormality constraints on the tangent and normal bases are kept by restricting the concatenated matrix to the Stiefel manifold and using Riemannian gradient descent with retraction; with the manifold fixed, the shape parameter is updated by a one-dimensional optimization.
Perspective
Experiments cover parameter estimation on synthetic spheres and a two-dimensional quadratic manifold, the effect of noise scale and neighborhood sample size on parameter estimation on Frey Face, face reconstruction on Isomap Face, and latent-space interpolation on Yale Face Database B; the method targets local quadratic manifold approximation, and readers should judge its applicability on that basis.
Shape-parameter estimation accuracy improves as ambient dimension and sample size grow, and bias is more visible at small noise scales; the authors list more general structured and dependent residual models, statistical consistency of the shape-parameter estimator, and stronger convergence guarantees for the alternating optimization as future work.
