Skip to main content
Search

# BLIP-Live

2 related articles

Subscribe to updates
  1. BLIP-Live makes when-to-speak part of the intelligence: on 326 ego-centric videos it cuts unnecessary responses by more than half versus state-of-the-art models

    Salesforce AI Research introduces BLIP-Live, a video agent that treats when to speak as part of the intelligence itself rather than only recognizing what is happening on screen. Earlier video agents combined live camera and voice interaction with product-grounded knowledge to guide complex procedures, but when a visual assistant should speak remained a complementary research question. BLIP-Live trains on large-scale data containing both useful response moments and periods when the agent should stay quiet, and uses Agent Memory to reduce repetition. Across 326 ego-centric videos, it reduced unnecessary responses of state-of-the-art models by over 50%, under an evaluation that charges for repeated and out-of-window messages and gives no credit for silence.

  2. arXiv

    Streaming video agents learn when to stay silent: BLIP-Live's PAUC-F curbs repeat firing, and its Qwen model averages 49.08 on ProactiveVideoQA, 44% above the prior best

    BLIP-Live reframes proactive streaming video as a causal decision to speak when evidence suffices and stay silent otherwise, and aligns data, metric, inference harness and training around that goal. It builds BLIP-Live-780K, a dataset of 784,944 human-verified responses annotated for both content and timing intervals, and proposes PAUC-F, a metric that balances response timeliness with firing precision. The trained BLIP-Live-Qwen model reaches state-of-the-art results on four proactive streaming benchmarks, averaging 49.08 on ProactiveVideoQA against 34.01 for the prior best MMDuet2, a 44% gain.

You’re all caught up