Skip to main content
Back to timeline
arXivSource publication:

Frozen RiskWorld stops picking trajectories by fixed rules: an 11,725-parameter ranker cuts nuScenes open-loop average L2 by 3.99%

Synopsis

Autonomous-driving world models predict the future of candidate trajectories, but choosing one still relies on handcrafted thresholds and lexicographic priorities fixed before training. A residual-guided reinforcement learning method trains candidate selection and bounded trajectory residuals jointly with PPO on top of the frozen RiskWorld world model and a fixed candidate bank, keeping only an 11,725-parameter ranking network at inference. In open-loop evaluation over 150 nuScenes validation scenes, average L2 falls from 0.405 m for the official RiskWorld baseline to 0.389 m, a 3.99% relative reduction; under matched feedback-access and actor-update budgets, residual exploration cuts L2 by a further 0.83% over categorical PPO without residuals.

Source-provided article image: Residual-Guided Lightweight Reinforcement Learning for Risk-Aware Trajectory Selection with a Frozen World Model
arXiv

Interpretation

A trajectory selection layer can learn scene-dependent candidate preferences without touching the world model or the candidate generator, adding only an 11,725-parameter ranking network at deployment. World models such as RiskWorld combine occupancy evolution with structured risk representations, yet picking a trajectory from them still relies on handcrafted thresholds and lexicographic priorities whose trade-offs are fixed before training. In open-loop evaluation over 5,119 contexts from the 150 nuScenes validation scenes, average L2 falls from 0.405 m for the fixed official RiskWorld baseline to 0.389 m, a 3.99% relative reduction.

During training the policy samples bounded residuals around the selected candidate and uses feedback from those local trajectories to update shared ranking features, while inference still returns the stored coordinates of an original candidate. Earlier PPO, A3C and expected-return methods used feedback only from original candidates and did no local exploration near them. At matched budgets of 9,830,400 feedback accesses and 800 actor updates, the residual-exploration version lowers average L2 by a further 0.83% over categorical PPO without residuals, and the average collision-rate point estimate falls from 0.101% to 0.098%.

At inference the selector adds the learned preference rank to the collision-risk rank from the frozen world model with fixed weights and takes the valid candidate with the smallest weighted sum. Fixed selection rules cannot weigh learned preference against predicted risk on a common ordinal scale. With risk features zeroed, average L2 is 0.5698 m for this method versus 0.6399 m for RiskWorld; with risk features shuffled it is 0.4636 m versus 0.5004 m, so the point estimates stay lower under both perturbations.

Perspective

Evaluation is confined to the open-loop setting on the nuScenes validation set, with annotated actor states and maps as inputs, and covers a single backbone and a single dataset; the source proposes reactive closed-loop evaluation of executed driving and interactions next, along with tests across more backbones and datasets using perception-derived inputs to assess transfer and robustness.

Open-loop metrics are measured against recorded actor trajectories and say nothing about closed-loop execution; the average L2 gain is modest; zeroing or shuffling risk features raises error for both methods, so performance still depends on the risk input.