MBZUAI presents Omni-Embed-Mini, which recasts cross-modal alignment as self-distillation through a shared frozen backbone: each media sample is paired with a dense cascaded caption and the teacher target is simply that same frozen backbone's embedding of the caption, so only projectors and phased LoRA adapters are trained; the 0.9B variant maps text, speech, audio, images, video and visually-rich documents into one cosine space while keeping text weights bit-identical, scoring 49.57 nDCG@10 on MTEB-v2 BEIR-8, and the 2.3B variant edges ahead of the closed gemini-embedding-2 on the overall-modality average, 51.39 to 49.51.
MBZUAI presents Omni-Embed-Mini, which recasts cross-modal alignment as self-distillation through a shared frozen backbone: each media sample is paired with a dense cascaded caption and the teacher target is simply that same frozen backbone's embedding of the caption, so only projectors and phased LoRA adapters are trained; the 0.9B variant maps text, speech, audio, images, video and visually-rich documents into one cosine space while keeping text weights bit-identical, scoring 49.57 nDCG@10 on MTEB-v2 BEIR-8, and the 2.3B variant edges ahead of the closed gemini-embedding-2 on the overall-modality average, 51.39 to 49.51.