Organizations: RIKEN Center for Advanced Intelligence Project (AIP), Japan · RIKEN Center for Interdisciplinary Theoretical and Mathematical Sciences (iTHEMS), Japan · Department of Mathematics, Keio University, Japan · National Institute of Technology, Matsue College, Japan · Institute of Systems and Information Engineering, University of Tsukuba, Japan
Diffusion language models generate text by reversing a classical diffusion process over token sequences. We show that their continuous-time forward and reverse dynamics can be embedded in the framework of open quantum systems, which describes quantum systems interacting with their environment. This connection provides a basis for extending diffusion sampling to superpositions of sentence states and for modifying the underlying generation dynamics. We construct a τ-leaping unraveling scheme that simulates stochastic trajectories of these states while updating multiple token positions at each step. This enables generation conditioned on superposed prompts or thoughts, which we call Superposition of Prompts (SoP) and Superposition of Thoughts (SoT). The same formulation yields Quantum-Inspired Remasking (QIR): perturbing the masked-diffusion dynamics produces a confidence-based rule for revising previously generated tokens. Experiments with pretrained diffusion language models show that superposition-based generation can retain information from conditioning components not selected at the final measurement, while QIR improves LLM-assessed text quality when applied at an appropriate stage of generation.
Figures & tables
Figure 1: A superposition of sentence states viewed as an open quantum system interacting with its environment.
Figure 2: Sampling by stochastic unraveling followed by projective measurement. The ensemble of pure-state trajectories reproduces the corresponding density-operator dynamics, while the final measurement yields a discrete token sequence.
Figure 3: Cosine similarity of SEDD-small outputs to forest-only (horizontal) and snow-only (vertical) reference texts, computed using MiniLM embeddings. Green and blue show single-condition baselines; gray shows SoP outputs with forest (left) or snow (right) selected at final measurement. Superposition increases average similarity to the unselected condition.
Prompt
Write a short story.
Thoughts
[forest] The narrative must take place in a dense, vibrant summer forest . […] The tone should feel alive, earthy, and teeming with natural energy . [snow] The narrative must take place in a desolate, frozen winter tundra . […] The tone should feel silent, harsh, and starkly isolated .
As the overcliff dawn drifts from sprawling forest of trees north – I stand on the edge of the frozen world , adrift. […] The air itself toys him away, like an elemental furnace or pine-scented forest earth . Snow piling up over the world, carrying liquid silver , and where the sun’s frame unfurls a chill ; dimming.
Table 1: Representative generated samples by iLLaDA-8B-Instruct ( Nie et al. (2026) ) with SoT . Coloring in the generated texts is generated by gemma-4-26B-A4B ( Gemma Team et al. (2026) ).
Model
Metric
none
end
middle
start
SEDD(medium) Lou et al. (2024)
GenPPL
33.12±0.58
32.73±0.57
27.52±0.43
12.24±0.26
Entropy
7.57±0.02
7.57±0.02
7.52±0.03
6.42±0.05
CoLA
0.285±0.006
0.285±0.006
0.335±0.006
0.336±0.007
Qwen (%)
–
51.5±0.8
62.0±1.3
53.3±1.9
ΔR7 (pp)
0.000±0.000
−0.005±0.004
0.025±0.013
1.405±0.210
RADD(medium) Ou et al. (2025)
GenPPL
33.66±0.60
33.34±0.59
28.08±0.44
13.71±0.28
Table 2: Results at γ=0.02 . Each condition contains 256 sequences for SEDD and RADD and 128 for iLLaDA. The no-remasking column is the shared control. All entries give mean ± standard error; Qwen preference scores are expressed as percentages, and ΔR7 in percentage points (pp). For ΔR7 , the no-remasking baseline is zero by definition.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Case Name
Index
Prompt Content
story __forest_snow
i=1 (forest)
“Write a short story. In the forest, the tall trees are everywhere. Green leaves cover the ground and the air is warm. Small animals run through the grass. There are many plants and flowers in this place. The woods are full of life and nature.”
i=2 (snow)
“Write a short story. In the deep snow, everything is white and cold. The ground is covered in thick ice and the wind is freezing. No plants are growing here. It is a very quiet and winter world with high mountains of snow.”
key __forest_city
i=1 (forest)
“Write a short story about a traveler searching for a lost key. The story takes place in a dense forest. The traveler searches among trees, follows animal tracks, and hears birds overhead.”
i=2 (city)
“Write a short story about a traveler searching for a lost key. The story takes place in a crowded city. The traveler searches among buildings, follows street signs, and hears traffic nearby.”
package __past_future
i=1 (past)
“Write a short story about a messenger delivering a mysterious package. The story takes place in a medieval kingdom. The messenger travels by horse, carries a sealed letter, and seeks a castle.”
i=2 (future)
“Write a short story about a messenger delivering a mysterious package. The story takes place in an interstellar civilization. The messenger travels by spacecraft, carries an encrypted message, and seeks a space station.”
Appendix
Table 3: Detailed prompt pairs used for SoP experiments.
Case Name
Index
Prompt Content
story_reasoning __forest_snow
i=1 (forest)
“ <chat> Write a short story. </chat> <think> The narrative must take place in a dense, vibrant summer forest. I need to emphasize overflowing life, humid warm air, lush green canopy, and the constant sounds of rustling leaves and active wildlife. The tone should feel alive, earthy, and teeming with natural energy. </think> ”
i=2 (snow)
“ <chat> Write a short story. </chat> <think> The narrative must take place in a desolate, frozen winter tundra. I need to emphasize absolute stillness, sub-zero cold, blinding white snowstorms, and a landscape completely devoid of plant life. The tone should feel silent, harsh, and starkly isolated. </think> ”
Appendix
Table 4: Detailed prompt pairs used for SoT experiments.The specific tags, like <chat> and <think> , used are replaced depending on the model.
Generation condition
N
Forest reference
Snow reference
Forest only
512
0.5367±0.0047
0.3898±0.0045
Superposition, ⟨\uppsi0∣forest⟩
185
0.5053±0.0074
0.4477±0.0082
Snow only
512
0.3898±0.0045
0.5439±0.0047
Superposition, ⟨\uppsi0∣snow⟩
327
0.4294±0.0062
0.5252±0.0063
Appendix
Table 5: Average cosine similarity for the forest/snow experiment using MiniLM embeddings. Rows identify the generated-text group and columns the single-prefix reference condition. Superposition rows denote outputs from \uppsit=0 with the indicated component selected; N is the group size. Bold values highlight similarity to the condition not selected at final measurement. The sample counts and uncertainties are those reported for each group.
Figure 4: Component weight p(i=1) for masked and uniform SEDD at two coefficient-update exponents. Rows show forest/city( key__forest_city in Table 3 ) and past/future( package__past_future in Table 3 ); each panel contains 120 trajectories. Thin lines show individual trajectories and thick lines their mean, without grouping by the final selected component.
Figure 5: Mean trajectory-level Shannon entropy for the same SEDD experiments with forest/city( key__forest_city in Table 3 ) and past/future( package__past_future in Table 3 ). Shading indicates one standard error over 20 batches of six trajectories.
Figure 6: Text-quality metrics at γ=0.02 . Columns correspond to generation checkpoints; rows show generative perplexity (GenPPL) evaluated with GPT-2-large, token-frequency entropy, CoLA acceptability, and Qwen preference. CoLA uses truncated classifier inputs (Appendix G.2 ). Higher Qwen preference favors remasking over no remasking. Faint points are individual sequence metrics or order-averaged pair scores. Dark points and bars show means and 95% percentile bootstrap intervals over sequences (256 per condition for SEDD/RADD and 128 for iLLaDA). GenPPL uses a logarithmic axis. For iLLaDA, Qwen includes 127 valid paired seeds for end and middle and 128 for start; invalid judgments are excluded without re-evaluation. The dashed line marks 50% preference; no self-comparison is evaluated for none .
Figure 7: Excess long-token-sequence repetition. Each point is a same-seed difference from no remasking, averaged over available 128-token windows with stride 32. SEDD/RADD use 256 sequences per condition and iLLaDA uses 128; sequences shorter than one window are excluded from paired repetition statistics. Bars are 95% bootstrap intervals over sequences, not windows. Values are in percentage points. All panels use the same vertical range, −0.5 –6 percentage points, to resolve differences in the means. Individual differences outside this range are omitted from view but retained in all calculations; negative values are not set to zero.
Masked diffusion language models (dLMs) generate text through bidirectional denoising, yet this capability remains locked for infilling prompts. This limitation is an artifact of the current supervised finetuning (SFT) convention of applying response-only masking. To unlock this capability, we extend full-sequence masking during SFT, where both prompts and responses are masked jointly. Once unlocked, the model infills masked portions of a prompt template conditioned on few-shot examples. We show that such model-infilled prompts match or surpass manually designed templates, transfer effectively across models, and are complementary to existing prompt optimization methods. Our results suggest that training practices, not architectural limitations, are the primary bottleneck preventing masked diffusion language models from infilling effective prompts
Diffusion language models intrinsically fail to capture correlations between decoded tokens, which leads to a harsh trade-off between sampling quality and throughput. To solve this issue, we propose DiLaDiff, a variant of masked diffusion language models with three components: (1) a continuous latent space with semantic capabilities, learned by an auto-encoder fine-tuned from an existing masked diffusion language model; (2) a latent diffusion model learning the prior over the encoder distribution; (3) a consistency model distilling the learned prior into a few-step latent generative model. We show that, even without distillation, our latent-guided diffusion model outperforms the masked diffusion baseline while significantly accelerating inference. Consistency distillation further lowers the computational overhead of continuous diffusion, such that the latent is generated in negligible time compared to discrete decoding.
Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3
Latent diffusion models offer an attractive alternative to discrete diffusion for non-autoregressive text generation by operating on continuous text representations and denoising entire sequences in parallel. The major challenge in latent diffusion modeling is constructing a suitable latent space. In this work, we present the Latent Diffusion Language Model (LDLM), in which the latent encoder, diffusion model, and decoder are trained jointly. LDLM builds its latent space by reshaping the representations of a pre-trained language model with a trainable encoder, yielding latents that are easy to both denoise and decode into tokens. We show that naive joint training produces a low-quality diffusion model, and propose a simple training recipe consisting of an MSE decoder loss, diffusion-to-encoder warmup, adaptive timestep sampling, and decoder-input noise. Ablations show that each component substantially impacts generation performance. On OpenWebText and LM1B, LDLM achieves better generation performance than existing discrete and continuous diffusion language models while being 2-13× faster, indicating that jointly learning the latent space is a key step toward making latent diffusion competitive for text generation.
Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4
Constructor University, Bremen, Germany · HSE University, Moscow, Russia · Applied AI Institute, Moscow, Russia +2