Skip to main content
Back to timeline
arXivSource publication:

OccSplat fills the geometry hidden behind occluders: surround-view single-frame reconstruction reaches 23.50 dB PSNR on nuScenes

Synopsis

Feed-forward Gaussian splatting reconstructs driving scenes directly from surround-view images, but it places a Gaussian only where an observed pixel exists, so regions uncovered as the vehicle moves forward appear as holes and stretched surfaces. OccSplat reframes this as occlusion-aware Gaussian densification: dual-background rendering locates regions lacking Gaussian support, and new 3D Gaussians are predicted for them and merged into the original representation. On the nuScenes validation set, OccSplat reaches 23.50 dB PSNR versus 21.76 dB for DrivingForward, with LPIPS falling from 0.194 to 0.182.

Source-provided article image: Quo Vadis, Driving Scene Reconstruction? Occlusion-Aware Gaussian Densification
Fig. 1 · arXiv · Page 2

Interpretation

OccSplat treats the gap in driving-scene novel-view synthesis as missing 3D geometry rather than patching rendered images. Earlier pixel-aligned feed-forward methods place a Gaussian only where an observed pixel can be lifted, so surfaces hidden at the current frame behind trees, poles, fences or building edges receive no primitives. On the nuScenes validation set, OccSplat reaches 23.50 dB PSNR, 0.767 SSIM and 0.182 LPIPS, against 21.76 dB, 0.767 and 0.194 for DrivingForward.

The method adds one feed-forward densification pass on top of the base Gaussian representation, first localizing unsupported regions and then generating and fusing new primitives for them. Gaussian Occlusion Mapping judges insufficient coverage from the difference between two renders of the same representation under complementary background colors, requiring no separate segmentation network or learnable parameters. Gaussian Occlusion Completion first restores structure in the preliminary render with a U-Net pixel refiner, then infers geometry and appearance in separate linear-attention depth and feature branches, reusing the Gaussian head trained during base reconstruction to emit new primitives.

Densification has to be confined to regions that genuinely lack support, or it adds redundant Gaussians in already reconstructed areas. Applying completion across the whole image without localization lowers PSNR from 21.76 dB to 19.92 dB and SSIM from 0.767 to 0.747. Occlusion mapping alone, without the refiner, already lifts PSNR to 22.27 dB; all three components together reach 23.50 dB, with the occlusion threshold best at 0.7, 22.91 dB at 0.99 and 22.55 dB at 0.5.

Perspective

Evaluation is confined to the nuScenes validation set at resolution in the single-frame surround-view setting, where only the current frame is observed and the next frame's surround views are synthesized; the work addresses researchers and engineers working on feed-forward driving-scene reconstruction.

The pixel refiner operates in image space and cannot explicitly enforce cross-view 3D geometric consistency, so severe disocclusion or large ego-motion revealing previously unobserved content may cause texture over-smoothing; the authors propose adding explicit 3D geometric and semantic priors and temporal information from consecutive frames.