Tactile post-training stops reading only instantaneous pressure: CoRA-VTLA recalls recent contact responses and lifts MuJoCo mean success from 50% to 73.33%
Synopsis
Pretrained vision-language-action models need post-contact correction in precision assembly, yet a single tactile reading cannot tell a tool pressing against an obstruction from contact that has settled. CoRA-VTLA adds a queryable action-response memory on the action-expert side, retrieving recent load changes conditioned on current contact and robot state, and uses contact-guided gaze supervision to pull a tactile semantic token's visual attention toward fingertip contact regions. Across three MuJoCo tasks it raises mean success from 50% for direct tactile fusion with the same sensor inputs to 73.33%, reaches 67% on the UniVTAC optical-tactile tube task, and averages 66.7% across three real-robot tasks.
Interpretation
Tactile post-training shifts from injecting current pressure into the policy to letting the policy query how recent contact responded to executed actions, so a pretrained vision-language-action model can decide after contact whether to keep pushing or back off and realign. Direct tactile fusion previously encoded current fingertip pressure and wrist force/torque into one action-side token, reaching 50% mean success across three MuJoCo tasks with the same sensor inputs. CoRA-VTLA succeeds in 110 of 150 trials across the three MuJoCo tasks, 73.33% on average; the 95% paired bootstrap confidence interval for the difference over direct fusion is 14.67 to 32 percentage points, with two-sided exact McNemar tests and Holm correction within each benchmark.
The action expert receives two tactile tokens, one describing the current load and one retrieving recent responses from memory conditioned on current contact and pose, and generates action chunks together with the vision-language context. The memory records changes in fingertip pressure and wrist force/torque after each executed action and keeps the wrist wrench relative to its initial reading to retain sustained loading; the query combines the current contact feature with robot state and is read out after a temporal Transformer encodes the events. In MuJoCo the most recent events span 0.8 s; removing the memory token drops mean success to 54%, and removing response-pretraining initialization also gives 54%, against 73.33% for the full configuration.
The tactile semantic token's visual attention is explicitly steered toward the image region where fingertip contact projects, avoiding the broader attention spread that direct tactile fusion produces in the action expert. On 24 matched MuJoCo observations, direct tactile fusion raises the normalized spatial entropy of the first action token's main-camera attention at layer 17 from 0.838 to 0.884 and lowers the contact region's share from 1.62% to 1.18%. With gaze supervision, the semantic token's mean attention share inside the projected contact region at VLM layer 17 rises from 2.77% to 46.65%, increasing on all 24 frames; in a real-robot tube-insertion example the front-camera share on the same frame rises from 2.6% to 12.6%.
Perspective
Evaluation covers three custom MuJoCo tasks (nail insertion, plug insertion, knob operation), three real-robot tasks (cup stacking, stamping, tube insertion) and Insert Tube on the public UniVTAC benchmark; each real-robot method is run in 50 trials per task, and each MuJoCo policy is evaluated on 50 unseen scenes per task, 150 trials in total. The memory window spans 0.8 s in MuJoCo, and UniVTAC uses a 0.8-second response history.
The ablation shows nail insertion falling to 26% without gaze supervision, so the mechanism's contribution varies across tasks; the authors list extending the memory horizon and improving the safety of recovery as future directions. Real-robot results come from 50 trials per task, and the UniVTAC result covers only Insert Tube.
