Cameras need not fire together: OffBeat changes only when each camera fires and beats the full 4DSloMo pipeline by 3.04 dB in dynamic-region PSNR
Synopsis
Multi-camera 4D reconstruction usually exposes every camera at the same instants, so fast motion between frames is lost. OffBeat treats a staggered rig as a spatially distributed multicoset sampler, derives from calibration alone which cameras fire together and in what order, and bounds a primitive's temporal window below by one camera frame interval. In simulated staggered capture on N3DV, it beats the full 4DSloMo pipeline with video diffusion by 3.04 dB in dynamic-region PSNR, and between the first and last synchronized frame it beats each scene's best synchronized method by 1.98 dB.
Interpretation
A camera rig can follow fast motion that synchronized capture misses by changing only when each camera fires, without raising the frame rate or adding cameras. A synchronized rig records once per camera frame interval, and adding cameras adds views of the same instants while temporal bandwidth stays at one camera frame rate; 4DSloMo already staggers, but its modular schedule fires each camera row in position order. On simulated staggered capture across six N3DV scenes, OffBeat's dynamic-region PSNR is 3.04 dB above the full 4DSloMo pipeline with video diffusion, and its full-frame PSNR is 4.35 dB above.
A row of cameras firing one after another in position order acts as one camera moving at constant speed, so depth and motion become confounded. This degeneration is the same depth-velocity ambiguity as watching a constant-velocity point with a single camera; repeated sweeps resolve it only for points that keep their velocity, not for fast non-rigid motion such as a flame or food being turned. For cameras on a line firing in sequence, the Fisher information is singular for any number of cameras; on N3DV only the vertical offset between the two rows keeps the information nonsingular.
Trigger groups and firing order can be computed from calibration alone, without recording any video. Earlier staggered schedules used a hand-set modular offset, without selecting groups by identifiability or ordering them by predictive variance for novel views. Grouping maximizes the working-volume average of the smallest eigenvalue of the information matrix, ordering minimizes predictive variance for novel views, and a local search finishes in seconds on a CPU; the modular schedule's predictive variance is 2.2 to 2.4 times OffBeat's on each N3DV scene.
A primitive avoids favoring any trigger group only if its temporal window lasts at least one camera frame interval. Spacetime Gaussians previously used a Gaussian temporal opacity, and a narrow window favors one group, letting that group's shared color or calibration error be fit exactly by content present only at those instants and appear as flicker at the camera frame rate in novel views. A proposition shows that windows weighting all groups equally for every number of groups and every center are exactly convolutions of a box of length one camera frame interval; the resulting closed-form smoothed box shows less flicker than a Gaussian at nearly equal PSNR.
Perspective
The demonstrated setting is simulated staggered capture obtained by subsampling 30 fps video, covering six N3DV scenes and six PanopticSports sequences with held-out camera 0 as the test view; the schedule needs calibration before capture. On PanopticSports between the first and last synchronized frame, OffBeat's dynamic-region PSNR is on par with synchronized capture plus interpolation.
On 4DSloMo's real staggered recording only the lifetime floor is tested, in its reconstruction stage, and no reliable gain is found; the period window in that stage lowers dynamic-region PSNR on one N3DV validation scene. OffBeat is more sensitive than synchronized STG to simulated trigger-time and photometric errors, which a real rig adds. Content shorter than one camera frame interval cannot be represented, and the fastest motion between a group's firings stays blurred.
