Egocentric video captions learn which hand does what: BiScopeNet lifts NVILA action recognition from 14.1% to 45.5%
Egocentric action captions usually say only that someone is picking up a pan, without saying which hand holds the pan and which holds the lid. BiScopeNet leaves the vision encoder and language model untouched, adding a text-guided attention module called FocusAlign at the projection stage between them and supervising training with MetaLoss, which masks metadata spans and upweights object words, alongside the video-level BiScopeCap dataset of hand roles and spatial metadata. On the full BiScopeCap set, the NVILA backbone with BiScopeNet reaches 45.5% action recognition, up from 14.1%, right-hand role recognition 44.9% from 17.2%, left-hand 41.2% from 28.7%, and object recognition 58.9% from 39.7%.