Skip to main content
Back to timeline
arXivSource publication:

Robot grasp planners stop scoring skills at observation time: indexing success by launch delay lifts success 5.1 points at 300 ms latency

Synopsis

Robots typically act 100 to 300 ms after the frame they observed, yet existing skill-success models predict success only for a launch at the moment of observation. Time-indexed skill success models take the launch delay as an input and predict whole-task success for a launch after that delay, with exact training labels from branched rollouts. In a MuJoCo UR5e task that intercepts a sliding puck, a receding-horizon planner using the model beat a static model by 2.9 to 5.3 points at 100 to 300 ms of latency and was better calibrated on delayed launches.

Source-provided article image: Time-Indexed Skill Success Models for Latency-Aware Skill Selection and Timing in Dynamic Manipulation
arXiv

Interpretation

Skill selection now treats when to launch as a predictable quantity: the planner no longer asks only whether a skill can succeed now, but estimates whether the whole task completes if that skill starts some milliseconds later. Earlier skill competence models, including initiation classifiers, affordance values and success discriminators, mapped the current state to the probability of success for an immediate launch, a launch time that latency makes impossible. In the MuJoCo study where a UR5e captures a sliding puck with three skills, the time-indexed TI-flat reached 59.8% to 48.4% success at 100 to 300 ms of fixed latency against 56.9% to 43.3% for the static model B1, and every paired 95% interval excluded zero.

The model learns labels for launches after a delay directly, instead of forecasting the future state and then applying a static model. The standard remedy rolls the state forward to execution time and evaluates a static competence model there; that is exact only in expectation over the forecast distribution, and point-forecast error grows with forecast spread relative to the skill's tolerance. Training stores a simulator snapshot at each time of a launch-delay grid, launches every skill from every snapshot and runs it to task end, yielding 2,666,595 exact (decision, launch) pairs; the planner re-solves over candidate waits every 0.1 s while the simulated clock keeps running during latency.

On delayed launches the time-indexed model's confidence matches observed success, while the static model is severely over-confident. The static model predicted near-certain success or failure on delayed launches and over-predicted InterceptGrasp success by 14.4 points on average; refitting a temperature halved its error but raised the zero-delay error from about 0.01 to between 0.13 and 0.23. Expected calibration error on delayed launches was 0.009 for the time-indexed model on InterceptGrasp and 0.006 on Block, against 0.239 and 0.234 for the static model; Brier scores were 0.075, 0.113 for the best forecast-based predictor and 0.241.

Perspective

The demonstrated setting is a MuJoCo UR5e capturing a sliding puck with three analytic skills, plus 49 test launches on a real UR5e that catches a rolling golf ball on an inclined board with a single skill; the intended reader is a robotics researcher working on skill selection and planning in dynamic manipulation.

The hardware test covers one skill, one delay and 49 launches on one day, labelled by one operator who also saw an automatic label, with the simulator calibrated on 35 separate launches; the time-indexed model was mildly under-confident on hardware, suggesting the board's disturbance on the test day was lower than the simulated level. In simulation the skills are analytic controllers, the robot holds its ready pose while waiting, the models interpolate between nine grid delays, and the state comes from a tracker rather than images.