Skip to main content
Search

# Progressive Multimodal Inference

1 related articles

Subscribe to updates
  1. Multimodal inference stops waiting for the slowest stream: AdaModalX cuts end-to-end latency by up to 64.75% across four mobile sensing tasks

    In distributed mobile sensing, multimodal foundation models usually wait for every modality to arrive in sync before inferring, yet data-heavy streams such as vision often lag audio, so waiting raises latency while inferring early loses accuracy. AdaModalX decouples inference execution from modality synchronization: a lightweight probe distilled from the edge-side model lets reliable samples exit on the device, and the edge decides when to activate the frozen foundation model based on candidate utility, imputing features from partially received delayed inputs when needed. Across four mobile sensing tasks, two inference architectures and four devices, it improves the accuracy-latency trade-off, with up to 26.97% accuracy improvement and up to 64.75% lower end-to-end latency.

You’re all caught up