D2ToF distills diffusion into one step: a single-frame single-photon waveform yields depth in about milliseconds, at roughly one-third the error of Marigold-DC
Synopsis
Low-cost single-photon LiDAR arrays deliver time-of-flight waveforms at very low spatial resolution, too coarse for direct depth imaging. D2ToF distills a multi-step conditional diffusion model into a one-step generator and adds a differentiable histogram data-consistency term to the training objective, so the reconstructed depth, re-rendered into low-resolution waveforms, matches the measurement; a high-resolution reflectivity image and a frozen monocular depth model supply geometric priors. On simulated Middlebury scenes under clean conditions the one-step model reaches an average RMSE of 11.7 mm, about one-third that of Marigold-DC at 32.9 mm, in a single forward pass of roughly milliseconds, far faster than its 250-step teacher.
Interpretation
Depth reconstruction from single-photon LiDAR moves from multi-step iterative sampling to a single forward generation, so even extremely low-resolution waveforms yield a high-resolution depth map in milliseconds. Earlier plug-and-play and iterative diffusion methods carry substantial inference cost and mostly rely on multi-frame video or a higher base spatial resolution, leaving efficient single-frame reconstruction at such low resolution insufficiently explored. On simulated Middlebury scenes in the clean setting (SBR 2.0, PPP 450), the one-step model averages 11.7 mm RMSE, below its 250-step DDPM teacher at 13.9 mm and below Marigold-DC at 32.9 mm and the histogram-aligned DepthAnything baseline at 33.0 mm.
Reconstruction is constrained by three signals at once: the time-of-flight histogram gives absolute range, the high-resolution reflectivity image gives spatial texture, and a frozen monocular depth model supplies the geometry the reflectivity image cannot resolve on its own. The histogram itself carries little spatial layout, while the reflectivity image is texture-rich but inherently ambiguous about 3D geometry, so neither signal alone recovers fine detail. In the ablation the full model reaches 11.71, 20.16 and 21.75 mm RMSE under clean, Noise 1 and Noise 2, below the 18.19, 25.83 and 26.69 mm of histogram plus reflectivity only and the 12.59, 21.69 and 23.11 mm of histogram plus monocular depth only.
During training the predicted depth is re-rendered through the single-photon forward model into a low-resolution histogram and compared with the measured one, anchoring the generated result to the acquired data. A conditional diffusion prior alone does not guarantee that a sample reproduces the observed photon-arrival histogram when passed through the physical forward model. Adding this consistency term lowers mean RMSE under all four simulated conditions, for example from 12.16 to 11.71 mm in the clean setting and from 21.26 to 20.16 mm under Noise 1 where only the reflectivity is degraded, although training uses clean reflectivity images only.
Perspective
Evaluation covers two operating points on simulated Middlebury scenes, clean and dark, plus real captures from an STMicroelectronics VL53L8 sensor under well-lit and low-light conditions; real data have no depth ground truth, so histogram consistency is used instead. The method requires a high-resolution reflectivity image co-registered with the SPAD array, and all training data are simulated waveforms from Hypersim and MPI Sintel.
Real data lack depth ground truth and are judged only by histogram consistency; under clean conditions the one-step student's surface normal error is 20.1 degrees, above the teacher's 18.1 degrees, despite its lower depth RMSE. Robustness to degraded reflectivity is measured zero-shot with clean guidance images in training, and performance on other sensors or larger arrays is not reported.
