Multimodal driving policies fail on timing, not just noise: STA lifts PPO collision rate from 16.8% to 35.6% in CARLA while using 17.8% fewer attacks
Synopsis
Multimodal reinforcement-learning driving policies depend on camera, LiDAR and vehicle-state observations staying aligned in time, and delaying one stream leaves the fused representation no longer describing a single instant. STA treats that misalignment as a unified attack surface: a history buffer replaces the current reading of a chosen modality with an older one, and a split-decision agent picks both when to attack and which modality to delay. In CARLA, against a PPO policy, STA raised the collision rate from 16.8% for the strongest baseline ADV to 35.6% while cutting executed attacks from 29.396 to 24.168, 17.8% fewer interventions.
Interpretation
STA treats cross-modal temporal misalignment itself as the attack surface rather than adding noise to one sensor, so a single perturbation scheme applies across heterogeneous inputs such as camera, LiDAR and vehicle state. Earlier adversarial attacks mostly relied on modality-specific perturbations, such as image noise or LiDAR point perturbations, which are tied to a particular sensor and poorly suited to probing the temporal-consistency vulnerability. In the dense-traffic Town02 scene of CARLA 0.9.13, against a PPO policy, STA raised the collision rate from 16.8% for the strongest baseline ADV to 35.6% while lowering the average number of attacks per episode from 29.396 to 24.168.
The attack runs through two branches: one decides whether to strike at the current step, the other decides which modality to delay, and the two are optimized separately under a finite attack budget. Earlier temporal perturbation methods mostly used random modality selection or fixed injection schedules, without using victim-policy feedback or accounting for an attack budget. The agent conditions on the victim policy's clean action and the remaining attack budget, is trained with PPO, and receives a sparse collision reward; on SAC, STA is also the strongest attack at 18.2% versus ADV's 16.8%, though the margin is smaller.
The effect of temporal misalignment depends on the victim policy, and a policy robust to spatial perturbations is not necessarily robust to timing. DARRL shows a 32.2% collision rate under the spatial attack ADV, higher than its 18.6% under STA, indicating that the two threat models expose different failure modes. Under the same Town02 configuration, STA delays image frames and drives a vision-language-model-based driving policy to a 62% collision rate, while the clean, random-delay and PGD conditions all stay at 0–2%.
Perspective
Evaluation runs in CARLA 0.9.13 on Town02 with dense traffic and fixed clear weather; victim policies receive an RGB image plus speed and waypoint states and output continuous steering and throttle, and each setting aggregates five independent seeds with 50 evaluation episodes per seed.
STA's delay is currently fixed to the oldest entry in the history buffer, and the authors list adaptive and learnable delay strategies as future work; the vision-language-model policy was evaluated in a single configuration, with broader architecture comparisons left open.
