Skip to main content
Search

# Egocentric Video Captioning

1 related articles

Subscribe to updates
  1. arXiv

    EgoCaptioner turns first-person video into timed steps, not one-line summaries, reaching 75.51 PSJ on EgoHome step alignment

    First-person video captioning has forced a trade-off between detail and timing: general captioners cover the scene without binding the narrative to the wearer's actions, while dense paragraphs leave unclear when each action begins. EgoCaptioner splits a caption into a video-level overview and timed steps, builds 50,000 training examples with a coarse-to-fine annotation pipeline that fixes event structure before expanding step descriptions, and refines the model with rewards for factual content, temporal alignment and style. Across five egocentric benchmarks it leads the evaluated open-source models in caption quality, caption-conditioned QA and temporal alignment, scoring 75.51 PSJ on EgoHome step segmentation and 78.84 Overall on PhysCapBench, above Gemini 3.1 Pro.

You’re all caught up