Diffusion and flow-based generative models have achieved remarkable results across diverse applications, but most assume data of fixed-dimension/length. Meanwhile, real-world problems such as layout generation often involve variable-length, multimodal data, such as continuous bounding box coordinates paired with discrete types. Existing methods typically address variable length by padding data to a fixed size or by introducing specialized objectives to model lengths explicitly, adding complexity and limiting generality. We propose Variable Length Diffuse Everything (VL-DE), a simple and general framework for variable-length multimodal generation on arbitrary state spaces. Instead of padding, VL-DE handles variable length through an insertion-based mechanism grounded in a principled theoretical framework, yielding a tractable training objective. In addition, by utilizing the fact that VL-DE is based on a generator with an additive form, we develop a higher-order sampler by appropriately handling the time dependence. Extensive experiments on layout and molecule generation demonstrate the effectiveness of our approach.
Figures & tables
Figure 1: Overview of VL-DE on a layout example. Variable-length data is generated by a joint process (Xt,St) , where St is an auxiliary process that tracks the active coordinates and Xt carries their values. Each coordinate i receives an insertion time τi , an insertion distribution and is denoised on its own clock u , so that all elements are clean simultaneously at t=1 .
RICO
PubLayNet
Model
FID ↓
Ali →
Ove →
FID ↓
Ali →
Ove →
Fixed number of slots, padded with an empty class
LayoutDM ∗
4.43
0.143
0.584
36.85
0.180
0.132
LayoutDiffusion
2.49
0.069
0.502
8.63
0.065
0.003
Number of elements given as input
DLT ∗
13.02
0.271
0.571
12.70
0.117
0.036
Table 1: Unconditional layout generation. Left: layout models grouped by how they handle the number of elements (numbers from Guerreiro et al. (2024) ; ∗ retrained there). Right: methods that generate the number of elements, on RICO, grouped by how they represent boxes. → : closer to the validation data is better. Protocol and sampling steps in Appendix E.1 .
Figure 2: Operator-splitting samplers on RICO (Mean over 5 sampling seeds, error bars one standard deviation). (a) Per step, the direct split is far ahead of Euler below 20 steps. (b) Per network evaluation, Euler is the most economical sampler at every budget.
Figure 4
Model
JSDm ↓
#Atoms
HBD
HBA
LogP
Weight
#C
#H
#O
#N
#F
Valid
Unique
Avg. 1− KS
Models with length access
ADiT
0.086
0.988
0.990
0.923
0.976
0.944
0.978
0.989
0.945
0.973
0.999
0.985
0.979
0.971
Multimodal FM
0.063
0.993
0.981
0.964
0.967
0.931
0.988
0.985
0.988
0.980
0.996
0.989
0.965
0.977
Variable length models
Transdimensional Jumps
0.120
0.870
0.866
0.832
0.833
0.883
0.815
0.856
0.925
0.855
0.968
0.978
0.986
0.870
Branching Flows
0.080
0.990
0.982
0.948
0.952
0.902
0.984
0.979
0.981
0.978
0.997
0.987
0.973
0.969
Table 3: Unconditional molecule generation on QM9. Multivariate Jensen–Shannon divergence (JSDm, lower is better), 1− KS distance to the data (higher is better) and fractions of valid and unique molecules; 10 K samples. Baselines from Billera et al. (2025) ;
Figure 4: Arithmetic equations. (a) The symbols and numbers of an equation; the number of symbols is set by the numbers and the parenthesis groups. (b, c) Joint distribution of these counts for the test data and 1000 samples of VL-DE. (d) ∣ left − right ∣ of the valid samples of VL-DE.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Inserted Object
Modalities
Arbitrary denoiser
Different per-modality length
Flex MDM
Mask tokens
Discrete
×
×
Transdimensional Diffusion
Noisy data
Euclidean
×
×
Edit Flows
Noisy data
Discrete
×
×
OneFlow
Noisy data
Euclidean + Discrete
×
×
Transfusion
Clean Tokens / Noisy Images
Euclidean + Discrete
×
×
Branching Flows
Duplicates
General
✓
×
Appendix
Table 4: Comparison of methods for modeling variable-length data. Inserted object : what is added when the dimensionality grows. Modalities : what state spaces are supported. Arbitrary denoiser : whether the per-coordinate dynamics can be any generator on V~ (as opposed to a fixed mask-diffusion or noisy-data scheme). Different per-modality length : whether each modality can carry its own independent length.
Figure 7: Error versus step size , minus the best error reached by any sampler (log-log). (a) RICO, FID . (b–d) QM9: one minus the average 1− KS, the fraction of valid molecules and the fraction of connected molecules ( 2 K molecules from one sampling seed per point). Dashed and dash-dotted lines have slopes 1 and 2 .
Hyperparameter
RICO
PubLayNet
Architecture
Transformer over the element set, no positional encoding
Layers
4
4
Width
512
512
Heads
8
8
Parameters
15.7M
15.7M
Insertion
from the prior, own clock
from the prior, own clock
Appendix
Table 5: Hyperparameters for layout generation on RICO and PubLayNet. The backbone matches the size of the Transformer used by Guerreiro et al. [2024] .
RICO
PubLayNet
Task
Model
FID ↓
Ali →
Ove →
mIoU ↑
FID ↓
Ali →
Ove →
mIoU ↑
Un-Gen
LayoutTransformer
24.32
0.037
0.542
0.587
30.05
0.067
0.005
0.359
LayoutFormer++
20.20
0.051
0.546
0.634
47.08
0.228
0.001
0.401
LayoutDM ∗‡
4.43
0.143
0.584
0.582
36.85
0.180
0.132
0.382
DLT ∗†
13.02
0.271
0.571
0.566
12.70
0.117
0.036
0.431
LayoutDiffusion ‡
2.49
0.069
0.502
0.620
8.63
0.065
0.003
0.417
Appendix
Table 6: Layout generation on RICO and PubLayNet, all tasks and methods. Baseline numbers are from Guerreiro et al. [2024] ;
RICO
PubLayNet
Task
Model
FID ↓
Ali →
Ove →
mIoU ↑
FID ↓
Ali →
Ove →
mIoU ↑
Completion
LayoutDM ∗
6.80
0.054
0.630
0.678
25.02
0.169
0.107
0.678
LayoutFlow
1.51
0.150
0.474
0.741
1.10
0.054
0.127
0.746
VL-DE (ours)
2.92
0.154
0.539
0.532
3.92
0.089
0.024
0.435
Refinement
RUITE
7.93
0.177
0.492
0.658
7.89
0.073
0.038
0.637
LayoutFormer++
3.67
0.141
0.503
0.656
2.94
0.042
0.013
0.642
Appendix
Table 7: Completion and refinement on RICO and PubLayNet. Baseline numbers are from Guerreiro et al. [2024] ; models marked with ∗ were retrained there on the common split. In completion, LayoutFlow and LayoutDM are given the number of missing elements, whereas VL-DE generates it.
Setting
FID ↓
Ali →
Ove →
epochs
Mechanism
Fully noised
2.34
0.217
0.557
–
Noisy data at the global time
2.30
0.221
0.543
–
Masked, then revealed
2.59
0.215
0.513
–
Schedule
tmax=0.25
2.55
0.240
0.497
–
tmax=0.5
2.43
0.233
0.509
–
tmax=1
11.94
0.269
0.645
–
Appendix
Table 8: Insertion mechanism and schedule on RICO , unconditional generation. Mechanism rows: tmax=0.5 , 5 sampling seeds with the sampler of the main results. Schedule rows: masked-then-revealed mechanism, single runs. Head rows: the same model at fixed length, where all elements are present from the start and revealed from the head, single runs; “epochs” is the number of training epochs to reach a validation FID of 2.5 .
Setting
FID ↓
Ali →
Ove →
Clocks
Independent category and box clocks
4.17
0.284
0.674
Shared clock, task mixture
2.58
0.169
0.497
Appendix
Table 9: Other ablations on RICO , unconditional generation. 5 sampling seeds.
Insertion weight wt
FID ↓
Ali →
Ove →
Length TV
1 (ours)
2.29±0.13
0.199
0.560
0.059
min(λt,50)
3.14±0.18
0.305
0.640
0.056
min(λt,50)/4.22
2.31±0.12
0.257
0.536
0.068
Appendix
Table 10: Hazard weight on the insertion loss on RICO , unconditional generation. Length TV is the total variation distance between the generated and test distributions of the number of elements. The unweighted row is a retrained copy of the model of the main results.
Figure 8: Unconditional layout generation. Rows are methods, columns eight layouts of a seeded batch; colors are categories. LayoutFlow is given the number of elements, the other methods generate it.
Model
FID ↓
Occ ↓
Rea ↓
Und l↑
Und s↑
Ove ↓
Histogram of N (canvas ignored)
VL-DE, canvas-blind
1.63
0.318
0.041
0.960
0.877
0.006
Fixed number of slots, padded with an empty class
VL-DE backbone, padding
4.02
0.144
0.022
–
0.774
0.013
Autoregressive with an end token
RALF without retrieval
2.89
0.125
0.019
0.967
0.917
0.011
Appendix
Table 11: Content-aware layout generation on CGL (test split, RALF’s evaluator), grouped by how the number of elements is modeled. RALF rows are computed from their released outputs; † marks numbers taken from the papers, which report no FID. “Proj.” applies a single containment projection at the last sampling step (Appendix E.4 ). Table 2 in the main text reports the four-column summary.
Figure 9: Content-aware layouts on CGL, unselected test canvases. Boxes are drawn over the inpainted canvas; underlays are drawn beneath the other elements.
Model
added-elem. err. ↓
added-text err. ↓
FID ↓
Layout, then strings (pipeline)
2.09
1.26
2.48
Joint (VL-DE)
1.96 †
1.06 †
2.34
Flat autoregressive sequence
1.81
0.89
8.54
Appendix
Table 12: Partial completion on Crello ( 1,691 test designs; real designs add 4.4 elements). Errors are mean absolute differences in the number of added elements and added text elements. † : significantly better than the pipeline (paired bootstrap over designs).
Figure 10: Partial completion on Crello , four test designs. Top: the given half of the design, in gray with its strings. Middle: the joint sampler of VL-DE adds elements (colored boxes) and writes their strings in the same run. Bottom: the real design. Designs were selected from test designs without a near-duplicate in the training set.
Backbone
EGNN, 9 layers, width 256
Head
Transformer, 4 layers
Parameters
9.0M
Insertion window
t∈[0,0.5]
Optimizer
AdamW, grad. clip 1
Learning rate
10−4 , 500 warmup steps
Schedule
cosine to 10−6
Appendix
Table 13: Hyperparameters on QM9.
Figure 11: Operator-splitting samplers on QM9 (average 1− KS, 2 K molecules per point). (a) Under fixed step size our samplers outperform Euler (b) Per network evaluation, all samplers obtain similar performance, with the best results coming from our samplers demonstrating improved accuracy.
Figure 12: Construction of an arithmetic equation. A target value is drawn (1), each side is a random expression tree with one open leaf, which is solved so that the side equals the target (2), and the rendered equation is split into its symbols and the magnitudes of its numbers (3).
The ability to generate variable-length proteins is crucial in protein design, where the optimal length is often unknown and tightly coupled to designability. Current diffusion- and flow-based generative models typically require the protein length to be specified before sampling, limiting their flexibility in exploring the feasible design space. To address this limitation, we introduce Generalized Poisson Flow (GPFlow), a variable-length generative framework that learns the rate function of an inhomogeneous generalized Poisson process by minimizing its negative log-likelihood. We establish population-level guarantees for recovering the joint multimodal distribution and derive an upper bound on the KL divergence between the data and generated distributions. We comprehensively evaluate GPFlow across structure and sequence design, motif scaffolding, and peptide co-design, spanning Euclidean, categorical, and Riemannian modalities to fully validate its variable-length generation quality. In unconditional design, GPFlow improves structural designability and achieves the best distributional fitness for sequence design compared to their corresponding fixed-length baselines, while perfectly recovering the length distribution. In conditional motif scaffolding, GPFlow ranks first on 10 of 16 structure-based design tasks with significantly more unique successes and also achieves more passed tasks in sequence-based design. In peptide co-design, GPFlow remains competitive even without access to a native-length oracle.
Chaoran Cheng, Zhanghan Ni, Yanru Qu +4
Siebel School of Computing and Data Science University of Illinois Urbana-Champaign
Latent Diffusion Models (LDMs) have become dominant in visual synthesis, but their quality-compute trade-off is largely constrained by the tokenizer's fixed compression ratio. Variable-length tokenizers (VLTs) promise adaptive compression by varying token counts, allowing diffusion models to flexibly balance quality and compute. However, conventional VLTs modulate length by truncating ordered token sequences, which makes token semantics depend on token position and breaks representational alignment across lengths. This leads to a cross-length shift in the latent distribution that hinders a single variable-length diffusion model from operating effectively. To address this, we propose a novel variable-length tokenizer that modulates length by merging tokens. We show that encouraging similar tokens to merge enables direct cross-length representation alignment when the diffusion transformer operates according to the merging pattern. Since conventional merging methods are data-dependent, making the merging pattern inaccessible during generation, we introduce learnable global merging, which is data-independent, to ensure compatibility with diffusion transformers. On ImageNet 256×256 generation, our merging-based variable-length tokenizer integrated with a diffusion transformer achieves a superior gFID-compute trade-off compared to prior VLT methods. Code is available at this https URL
Dong Hoon Lee, Seunghoon Hong
Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea · School of Computing, KAIST, Daejeon, South Korea
Diffusion language models (DLMs) have recently emerged as a promising alternative to autoregressive models, primarily due to their ability to enable parallel decoding. Despite this advantage, most existing DLMs rely on a fixed generation length specified prior to decoding, which restricts their flexibility in real-world applications. While a few recent works attempt to support flexible-length generation, they typically suffer from notable limitations: some require costly retraining to accommodate variable-length outputs, while others depend solely on local confidence signals during decoding. Such local criteria fail to capture the evolving structure of the sequence, often resulting in suboptimal generation quality. In this paper, we propose a training-free, Bayesian structured decoding framework that formulates flexible-length generation as a dynamic structural inference problem. Our approach formulates flexible-length generation as a dynamic structural inference problem, jointly computing the expansion length, the block boundaries, and the decoding schedule. At each window expansion step, the method integrates local uncertainty with structural signals via a unified mechanism that supports dynamic structured generation, including both flexible block expansion and block organization, while maintaining coherence. Extensive experiments across multiple benchmarks demonstrate that our approach significantly improves generation quality and flexibility over existing fixed-length and flexible-length baselines. These results highlight the advantage of Bayesian structured decoding for diffusion language model, providing a principled and efficient solution for structured text generation.