Skip to main content
Search

# Multimodal Continual Instruction Tuning

1 related articles

Subscribe to updates
  1. arXiv

    Multimodal continual tuning stops stacking experts: CoPre-MoE merges overlapping ones into a fixed pool, holding UCIT forgetting at -0.10

    When a multimodal large model learns tasks in sequence, adding a fresh set of experts per task avoids overwriting old parameters but makes expert count grow linearly and lets different tasks relearn overlapping functions. CoPre-MoE consolidates functionally overlapping experts into a fixed-capacity generic pool, factorizes the remaining task-specific experts into shared projection bases plus compact cores, and constrains routing with frozen historical retrieval keys and an orthogonality penalty. On the UCIT and CoIN continual instruction-tuning benchmarks it reaches the highest average score, with backward transfer of -0.10 on UCIT and 68.3% fewer adaptation parameters than MoELoRA.

You’re all caught up