TOMAS guesses the operation before the path: grasp, push or pull on one object, and the lowest error on all six 3D hand-forecasting metrics
In egocentric video, one object can be grasped, pushed or pulled, so past hand motion alone does not pin down where the hand goes next. TOMAS combines finger-joint topology, object semantics and spatial relations to infer candidate interaction targets and latent operations, then splits the future into ordered stages with learnable temporal boundaries and refines the trajectory with stage-specific geometric subgoals. On EgoPAT3D-DT, H2O-PT and EGTEA Gaze+, TOMAS reaches the lowest error on all six 3D metrics, tying Uni-Hand on H2O-PT FDE, and on five of six 2D metrics.