CLIP incremental learning stops relying only on [CLS] and [EOS]: PACE feeds patch and token detail into the decision and leads last accuracy on nine benchmarks
CLIP-based class-incremental learning has mostly matched the global visual [CLS] and textual [EOS] representations across modalities, leaving patch-level visual detail and token-level textual semantics out of the final match. PACE selects the top-k visual patches and textual tokens most related to the global representations from the frozen CLIP encoders, injects that local information into the global representations through a cross-modal fusion module, and constrains updates of the shared fusion weights with subspace projection. Across nine benchmark datasets, PACE reports higher last and average accuracy than the listed comparison methods, for example a last accuracy of 93.94 on Cars B0 Inc10.