Skip to main content
Search

# Synergistic Information

2 related articles

Subscribe to updates
  1. arXiv

    InfMasking masks features before fusion so no single modality dominates, lifting the synergy task 5.6 points over CoMM

    Multimodal fusion often leans on a dominant modality or on shared information, leaving unused the synergy that exists only when modalities are observed together. InfMasking randomly masks feature tokens of each modality before fusion and requires the masked fused representation to stay consistent with an unmasked view or a class proxy, turning the expectation over infinitely many masks into variational bounds under a Gaussian model. On the controlled Trifeature benchmark it reaches 77.0% on the synergy task, 5.6 points above CoMM, and 90.6% on uniqueness.

  2. arXiv

    Three-modality contrastive learning swaps its negatives: CHORD gains 21.2 points over the strongest baseline on samples that need all three modalities

    Higher-order objectives for three-modality contrastive learning are meant to capture information that appears only when all three modalities are seen together, but ordinary negatives are built by swapping modalities, so a single pairwise check rejects them and the higher-order term receives almost no gradient. CHORD instead draws decoy negatives from a calibrated pairwise model, negatives that model itself finds plausible, and trains a residual to separate them. On M3-Mix, where one in ten training samples needs all three modalities, CHORD improves accuracy on those samples by 21.2 percentage points over the strongest of eight baselines.

You’re all caught up