Skip to main content
Back to timeline
arXivSource publication:

EviBridge feeds predictive visual evidence into an interaction model, lifting tool-handover overall score from 0.2178 to 0.6448

Synopsis

In streaming video, an assistant should have a tool ready before the worker asks, and should cancel that preparation if the request is withdrawn. EviBridge routes features from a frozen V-JEPA 2 predictive encoder through a trainable interface into the language decoder while keeping observation timing and order, so the model jointly predicts interaction state and which tool to hand over. On FuGIBench's 1,000 synthetic tool-assistance videos, the best EviBridge-E variant, E-RecentHist, scores 0.6448 overall against 0.2178 for its task-adapted native baseline E-Native; the best EviBridge-I variant, I-MLP, scores 0.5892 against 0.5327 for I-Native.

Source-provided article image: EviBridge: Encoding Predictive Visual Evidence for Proactive Interaction
arXiv

Interpretation

A tool-handover assistant no longer waits for an explicit instruction before acting; it enters a preparation state as the observed action develops and keeps reassessing whether the handover still holds. Earlier streaming interaction models decided what to say only after interaction cues appeared, lacking a mechanism to prepare before a response is warranted and to cancel when the request is withdrawn. On FuGIBench's 1,000 evaluation videos, the best EviBridge-E variant E-RecentHist scores 0.6448 overall against 0.2178 for the task-adapted native baseline E-Native; the best EviBridge-I variant I-MLP scores 0.5892 against 0.5327 for I-Native.

The model sends predictive visual features into the language decoder alongside native visual inputs, and one autoregressive generation path outputs both the interaction state and the tool information. The frozen V-JEPA 2 encoder only extracts features; a trainable evidence interface maps them into decoder space while preserving observation attributes or order. EviBridge-E explicitly encodes type, age and spatial position of recent versus historical evidence, while EviBridge-I preserves slot order and turn order of pooled features and generates dedicated interaction tokens; both are evaluated on FuGIBench's target handovers, long waits and cancelled requests.

The gain from predictive evidence spans successful handovers, long stretches with no request, and requests that are withdrawn. Off-the-shelf baselines score at most 0.5000 on cancelled requests, and some activate frequently during annotated Wait intervals. On cancelled requests, E-RecentHist scores 0.9056 and E-Recent 0.9017 against 0.2115 for E-Native; I-MLP raises I-Native from 0.6157 to 0.8311.

Perspective

Evaluation is confined to FuGIBench's 1,000 synthetic tool-assistance videos, comprising 600 target handovers, 200 long waits and 200 cancelled requests, replayed at 4 FPS with decoding once per second; the conclusions concern the tool-handover interaction setting, and the authors propose future work on architectures that adaptively select and aggregate predictive visual evidence and on evaluation across broader scenarios and tasks.

Zeroing the age branch output in E-RecentHist drops overall score from 0.6448 to 0.3127 and preparation to 0.0037, showing this variant depends markedly on age encoding, while zeroing type or position changes overall score by less than 0.004. E-RecentHist's preparation score is lower than E-Native's, and E-Native's higher preparation comes with frequent activation during reference Wait intervals.