Skip to main content
Search

# Scale-Contaminated Training

1 related articles

Subscribe to updates
  1. arXiv

    Computing loss on scaled targets cuts MASE in all 24 architecture-benchmark comparisons of time series foundation model pretraining, by 18.8% on GIFT-Eval on average

    The work proves that reversing an affine scaling (e.g., ReVIN) before computing the loss multiplies each series' gradient by a power of the scaling denominator, letting high-scale series dominate training (ScaleCon); computing a homogeneous residual loss directly on scaled targets (ScaleIn) instead leaves every mini-batch gradient and the full optimization trajectory invariant to arbitrary independent positive rescaling of training series, and in pretraining across four TSFM architectures it lowers MASE in all 24 architecture-benchmark comparisons, by 18.8% on GIFT-Eval and 21.9% on the M-competitions on average.

You’re all caught up