ROSE keeps multimodal models from forgetting as edits pile up: orthogonal subspace updates hold VLKEB reliability at 95.24 after 1,000 edits
Synopsis
Multimodal large language models need continuous factual updates, yet sequential edits erode pretrained abilities and overwrite earlier changes. ROSE confines each weight update to the orthogonal complement of the existing knowledge subspace, uses a dynamic risk mask to protect parameters critical for general capability, and at inference decides by prototype similarity whether to activate the edited branch. On VLKEB, ROSE's reliability falls from 100.00 to 95.24 as edits grow from 1 to 1,000, while FT-L drops 26.15 points under the same setting.
Interpretation
Lifelong knowledge editing for multimodal large language models now has a scheme that separates writing new facts from protecting old capability: during editing, ROSE changes only parameter components orthogonal to existing knowledge directions. Earlier lifelong editors were either too conservative, weakening the reliability of new edits, or too intrusive, interfering with pretrained capability and prior edits, and they lacked a basis for identifying which parameter subspaces are vulnerable to degradation. On VLKEB, ROSE's reliability falls from 100.00 to 95.24 as edits grow from 1 to 1,000, while FT-L drops 26.15 points on the same benchmark and setting; at 1,000 edits ROSE exceeds LiveEdit in reliability by 17.85 points on E-IC and 4.66 points on E-VQA.
Beyond writing, ROSE decides at inference, coarse to fine, whether an input should trigger edited knowledge, reducing false activation on unrelated questions. Earlier modular methods relied on routing to localize knowledge but overlooked the parameter update trajectory, so they could not prevent subspace collisions when shared parameters were updated. At 1,000 edits on VLKEB, ROSE reaches 100.00 textual locality (T-Loc) and 88.39 multimodal locality (M-Loc); by contrast, SERAC's M-Loc on VLKEB falls to 15.04.
In implementation, ROSE freezes the backbone, places sequential updates in an auxiliary branch, and maintains separate subspace bases for visual and textual historical inputs. Visual and textual representations are entangled, making the knowledge space especially sensitive to sequential perturbation, so a single factual update can trigger cross-modal chain degradation. Ablations on VLKEB: removing multimodal decoupling gives M-Loc 40.54, removing knowledge fusion gives 84.37, and full ROSE gives 88.39; removing orthogonal editing gives reliability 78.71 against 95.24 for full ROSE.
Perspective
Experiments cover the image-captioning and visual question-answering tasks of MMEdit, plus VLKEB, which is built as a visual question-answering benchmark; the default backbone is LLaVA-v1.5-7B, with extensions to LLaVA-v1.5-13B, MiniGPT-4 and Qwen2-VL. Reported edit sequences reach 1,000 edits without capacity compression. Efficiency was measured on a single NVIDIA RTX 4090, at roughly 1.3 to 1.7 seconds per edit and 1.0 to 1.2 seconds per query, with peak VRAM rising from 16781.52 MB to 17005.69 MB.
When the accumulated historical feature subspace approaches full rank, strict orthogonality permits only a zero update, so ROSE must trade off against protecting all historical directions; SVD truncation is only a possible extension for this case, and the reported experiments do not enable capacity compression. The average metric is 92.18 at a risk-sensitivity threshold of 0.5, 92.22 at 0.9, and falls to 88.72 at 1.0, indicating a trade-off region for that threshold.
