Multimodal models swap coordinate text for a 1D vector codebook: DVD reaches 40.0% AP3D on SUN-RGBD and 499 ms per object
When multimodal large language models handle 2D and 3D perception, coordinates are either written as text, which costs many tokens, or quantized into a fixed integer range, which limits range and precision. DVD converts 2D boxes, 2D masks and 3D boxes into 1D vector sequences, maps them to a few discrete tokens through a shared codebook, and restores the original representation at inference with a lightweight de-tokenizer. On SUN-RGBD, Hypersim and nuScenes, DVD-2B reaches 40.0%, 15.7% and 19.8% AP3D, and on RefCOCOg val it reaches 89.9% Precision and 70.4% cIoU.