Skip to main content
Search

# Dynamic Vector Decoding

1 related articles

Subscribe to updates
  1. arXiv

    Multimodal models swap coordinate text for a 1D vector codebook: DVD reaches 40.0% AP3D on SUN-RGBD and 499 ms per object

    When multimodal large language models handle 2D and 3D perception, coordinates are either written as text, which costs many tokens, or quantized into a fixed integer range, which limits range and precision. DVD converts 2D boxes, 2D masks and 3D boxes into 1D vector sequences, maps them to a few discrete tokens through a shared codebook, and restores the original representation at inference with a lightweight de-tokenizer. On SUN-RGBD, Hypersim and nuScenes, DVD-2B reaches 40.0%, 15.7% and 19.8% AP3D, and on RefCOCOg val it reaches 89.9% Precision and 70.4% cIoU.

You’re all caught up