Skip to main content
Search

# Spectral Framework for Universal Proxy Regularization

1 related articles

Subscribe to updates
  1. arXiv

    Model merging drops peer-task data: with ImageNet as a universal proxy, merged CLIP models approach the oracle that sees other tasks

    Model merging builds a multi-task model from independently fine-tuned experts, but earlier pre-merging regularizers needed data from the other tasks to be merged, breaking the independence assumption. This work characterizes a universal proxy distribution against target tasks through spectral coverage and spectral leakage, and derives a unified bound in which smaller spectral parameters mean lower parameter interference. On CLIP vision benchmarks, three instantiations regularized with an ImageNet proxy beat existing pre-merging baselines and approach the oracle that uses peer-task data; on language models, a WikiText-103 proxy unrelated to TRACE also improves merged performance.

You’re all caught up