Skip to main content
Back to timeline
arXivSource publication:

Gaussian splatting blurs its own fine detail: AGSR contracts projected covariances into multi-aperture renders and lifts PSNR by 4.34 dB on degraded representations

Synopsis

3D Gaussian splatting synthesizes novel views in real time, but each Gaussian's projected footprint acts as a low-pass filter that erases fine boundaries and texture. AGSR contracts the projected covariances at inference time to produce a sequence of renderings with progressively smaller apertures, then fuses them with a network of only 21K to 139K parameters using uncertainty, transmittance and depth cues, as a plug-in for pre-trained 3DGS models. On Mip-NeRF 360 with training images downsampled 8x, AGSR beats bicubic upsampling by 1.69 dB PSNR and native 3DGS by 4.34 dB.

Source-provided article image: AGSR: Aperture-Guided Super-Resolution for Gaussian Splatting
arXiv

Interpretation

The high-frequency detail in a Gaussian-splatting render is limited by the screen-space footprint of each Gaussian, not by information missing from the scene representation. Earlier responses either densify by adding Gaussians, which raises memory, storage and rendering overhead, or re-optimize at the scene level with learned super-resolution priors, which adds optimization time and training cost. On Mip-NeRF 360, as training-image resolution drops from 2x to 8x downsampling, native 3DGS PSNR falls from 28.74 dB to 21.15 dB, and AGSR's gains grow as the representation degrades.

AGSR leaves the underlying Gaussian representation unmodified and undensified, upscaling low-resolution renders into high-resolution views only at inference. It uses covariance contraction to build a multi-aperture rendering sequence, where aperture means the effective screen-space footprint scale of the rendering primitives; smaller footprints widen the effective bandwidth, and structure erased by native rendering reappears as cross-aperture variation. Those variations are collected into a parameter-free structural uncertainty field marking likely under-resolved regions; the network then uses transmittance and positionally encoded depth as geometric gating, injects higher-bandwidth observations across three stages, and a sub-pixel head produces the high-resolution image.

When the representation is severely under-resolved, rendering at low resolution and then upsampling can beat direct high-resolution rasterization. Bicubic upsampling exceeds native 3DGS in PSNR and SSIM under severe degradation, because low-resolution rendering itself applies stronger low-pass filtering that suppresses unstable high-frequency artifacts and aliasing from the degraded representation. A Bonsai scene ablation raising the minimum projected footprint from 0.3 to 1.2 shrank the PSNR gap between native rendering and bicubic upsampling from 1.75 dB to 0.22 dB.

Perspective

Evaluation uses the Mip-NeRF 360 benchmark with the underlying 3DGS representation frozen; AGSR inputs are rendered one factor below a fixed evaluation target resolution, training images are downsampled by 2, 4 and 8 to produce checkpoints at different degradation levels, and super-resolution scales are 2x and 4x. The authors propose future work on adaptive aperture selection, improved runtime efficiency and stronger restoration under severely degraded representations.

At low degradation and 2x super-resolution, SAFMN reaches SSIM 0.830 and LPIPS 0.281, marginally better than AGSR's 0.827 and 0.287; the authors explain that when the native representation is already high-fidelity, covariance contraction exposes limited extra structure and gives weaker discriminative guidance. Inference speed falls as active stages increase: the 4x task with three stages runs at 64 FPS in FP32, and the 2x task with three stages at 22 FPS.