Label only a sliver of rewards: SURE multiplies successor reachability by reward uncertainty to approach full-feedback performance with far fewer labels
When only a tiny fraction of transitions in an offline dataset can receive reward labels, SURE multiplies two signals, which states can reach currently high-value states and where the reward model remains uncertain, to decide where the next label goes. Queried rewards are retained to fit a reward model that annotates the remaining transitions. On Maze2d-Medium, Maze2d-Large and HalfCheetah, the feedback fractions SURE needs to reach 95% of the full-feedback reference were the lowest among all methods, at 0.20%, 0.15% and 0.71% respectively.