Skip to main content
Back to timeline
arXivSource publication:

VL-DE drops padding for variable-length multimodal generation: FID 2.32 on RICO layouts, just ahead of LayoutFlow's 2.37

Synopsis

Diffusion and flow models mostly assume fixed-length data, yet layouts and molecules pair continuous coordinates with discrete types and vary in length with content. VL-DE skips padding to fixed slots and instead assigns each coordinate an insertion time, then denoises it on its own clock, generating variable-length multimodal data directly on arbitrary state spaces. On RICO layout generation it receives no element count and pads nothing, reaching FID 2.32 against 2.37 for LayoutFlow, with validation data at 2.10.

Source-provided article image: Variable Length Diffuse Everything: Arbitrary Length Generative Modeling on Arbitrary State Spaces
arXiv

Interpretation

Variable-length multimodal generation can proceed without padding and without length as an input, with the generative process defined on the original state space. Earlier approaches either padded data to a fixed number of slots or supplied the element count as an input or a separately modeled target, and multimodal variable-length methods often required all modalities to share one length. On RICO layout generation, VL-DE neither receives the element count nor pads, reaching FID 2.32 against 2.37 for LayoutFlow; on the same dataset Edit Flows scores 6.43, FlexMDM 8.41, OneFlow 3.31 and Branching Flows 12.09.

Each coordinate gets an insertion time and, once inserted, is denoised on its own clock so that all coordinates are clean at the end. Training augments the state with an auxiliary variable recording latent insertion operations, so sampling the joint process yields a tractable loss without summing over insertion histories that could produce the same state. On unconditional layout generation, VL-DE reaches FID 2.32 on RICO and 8.45 on PubLayNet, against validation data at 2.10 and 8.10.

Insertion and denoising can be integrated separately, giving a second-order sampler for time-dependent Markov processes. Standard operator splitting assumes the split terms carry no explicit time dependence, which fails here because both insertion and denoising vary with time. On the learned-initialization model, the direct split sampler beats Euler per step at low step counts and the two level off as steps increase; per network evaluation, Euler is better at every budget.

Perspective

The demonstrated scope is two-modality variable-length generation: unconditional layout generation on RICO and PubLayNet, content-aware layout generation on CGL, 3D molecule generation on QM9, and an arithmetic-equation toy problem whose symbol and number sequences differ in length; the Crello experiment with independently varying element counts and text-string lengths is explicitly described as a demonstration rather than a benchmark result.

The framework supports arbitrarily many modalities, but all experiments use two; on layouts, the sampler's gain per step does not carry over to a gain per network evaluation, and could become a significant cost advantage only where one operator is cheap to evaluate or caching is available.