3D generation skips the latent space: Direct3D-V3 diffuses directly on SDF voxels for finer mesh detail
Direct3D-V3 moves image-to-3D diffusion out of a compressed latent space and onto raw SDF voxel grids, so neither training nor sampling passes through a VAE. The dominant recipe first compresses geometry into vecset, triplane or sparse-voxel latents and then diffuses there, and detail lost in encoding cannot be recovered downstream. To make direct voxel modeling workable, the model splits the work between a patch-level and a voxel-level DiT and adds a neighbor-SDF smoothness constraint, differentiable Marching Cubes normal supervision and camera prediction; on 102 real, web-collected and AI-generated images its Uni3D and ULIP-2 consistency scores exceed the listed open-source methods, and the authors report finer local mesh detail.