IGRecon4D separates moving traffic from the static background, lifting full-image PSNR 2.58 dB over DGGT on Waymo
Synopsis
Feed-forward 4D driving-scene reconstruction has tended to absorb pedestrians and vehicles into the static representation and to let Gaussian positions drift across viewpoints. IGRecon4D distills an instance-level semantic branch into a dynamic-mask prior during training and anchors Gaussian positions with dense depth recovered from sparse LiDAR. On the Waymo Open Dataset it reaches 29.51 dB full-image PSNR, 2.58 dB above the strongest baseline DGGT, and lowers LPIPS from 0.116 to 0.102; in zero-shot nuScenes it leads on traffic-participant-region metrics but not on full-image metrics.
Interpretation
IGRecon4D splits feed-forward 4D driving-scene reconstruction into two coupled questions: which pixels belong to potentially moving traffic participants, and where each Gaussian should sit. Earlier feed-forward reconstruction methods were built mainly for static scenes, while feed-forward dynamic methods such as DGGT still struggle to combine accurate geometry, multi-view consistency and traffic-participant-region localization in complex driving scenes. On the Waymo Open Dataset validation split, IGRecon4D reaches 29.51 dB full-image PSNR against 26.93 dB for DGGT, a 2.58 dB gap; LPIPS falls from DGGT's 0.116 to 0.102, and SSIM is 0.869 versus 0.868.
The dynamic mask is not learned from motion labels; an instance semantic branch sharing features with reconstruction supplies a prior that a consistency constraint passes to the dynamic-mask head. That branch predicts instance masks through learnable queries in two training stages: the first trains it on pseudo instance annotations, and the second freezes it and uses the union of its predicted instance masks as the supervision target for the dynamic mask. In the ablation, removing the constraint and initializing the dynamic head from DGGT instead drops traffic-participant-region PSNR from 26.76 to 23.81 and that region's SSIM from 0.858 to 0.774.
Gaussian positions are not regressed directly from images; they come from back-projecting dense depth that a pretrained network recovers from LiDAR projected onto the image plane. Directly regressing 3D positions is unstable in large driving scenes, and sparse LiDAR covers only some pixels, so dense depth must be recovered first. In the ablation, replacing LiDAR-guided depth with the DPT-style dense depth prediction used by DGGT lowers full-image PSNR from 29.51 to 25.20, raises LPIPS from 0.102 to 0.213 and raises D-RMSE from 4.610 to 11.584.
Perspective
Evaluation covers 202 scenes from the Waymo Open Dataset validation split plus zero-shot testing on nuScenes; input is three front-facing cameras, sparse LiDAR and 20 consecutive frames, with inference on a single RTX Pro 6000. The authors list longer temporal windows, broader camera configurations and larger-scale scenes as future directions.
The instance-level prior is semantic rather than motion-state supervision, so it cannot distinguish static from moving objects of the same category, such as parked and moving vehicles. On zero-shot nuScenes, AnySplat obtains better full-image PSNR, SSIM and LPIPS, with IGRecon4D's advantage concentrated in traffic-participant regions. The authors also report that a nighttime scene lit by streetlights was not reconstructed with the illumination cast by those lights.
