Diffusion Language Models as Open Quantum Systems

Akinori Tanaka; Yukinao Akamatsu; Takuki Tomita; Yuji Hirono

cs.LGOct 15, 2026

Diffusion Language Models as Open Quantum Systems

Authors: Akinori Tanaka, Yukinao Akamatsu, Takuki Tomita, Yuji Hirono

Organizations: RIKEN Center for Advanced Intelligence Project (AIP), Japan · RIKEN Center for Interdisciplinary Theoretical and Mathematical Sciences (iTHEMS), Japan · Department of Mathematics, Keio University, Japan · National Institute of Technology, Matsue College, Japan ·

Abstract

Diffusion language models generate text by reversing a classical diffusion process over token sequences. We show that their continuous-time forward and reverse dynamics can be embedded in the framework of open quantum systems, which describes quantum systems interacting with their environment. This connection provides a basis for extending diffusion sampling to superpositions of sentence states and for modifying the underlying generation dynamics. We construct a $τ$ -leaping unraveling scheme that simulates stochastic trajectories of these states while updating multiple token positions at each step. This enables generation conditioned on superposed prompts or thoughts, which we call Superposition of Prompts (SoP) and Superposition of Thoughts (SoT). The same formulation yields Quantum-Inspired Remasking (QIR): perturbing the masked-diffusion dynamics produces a confidence-based rule for revising previously generated tokens. Experiments with pretrained diffusion language models show that superposition-based generation can retain information from conditioning components not selected at the final measurement, while QIR improves LLM-assessed text quality when applied at an appropriate stage of generation.

Figures & tables

Figure 1: A superposition of sentence states viewed as an open quantum system interacting with its environment.

Figure 2: Sampling by stochastic unraveling followed by projective measurement. The ensemble of pure-state trajectories reproduces the corresponding density-operator dynamics, while the final measurement yields a discrete token sequence.

Figure 3: Cosine similarity of SEDD-small outputs to forest-only (horizontal) and snow-only (vertical) reference texts, computed using MiniLM embeddings. Green and blue show single-condition baselines; gray shows SoP outputs with forest (left) or snow (right) selected at final measurement. Superposition increases average similarity to the unselected condition.

Prompt	Write a short story.
Thoughts	[forest] The narrative must take place in a dense, vibrant summer forest . […] The tone should feel alive, earthy, and teeming with natural energy . [snow] The narrative must take place in a desolate, frozen winter tundra . […] The tone should feel silent, harsh, and starkly isolated .
Generated $\braket{\bm{\uppsi}\|\textbf{\color[rgb]{0,0.4688,0.2734}forest}}$	As the overcliff dawn drifts from sprawling forest of trees north – I stand on the edge of the frozen world , adrift. […] The air itself toys him away, like an elemental furnace or pine-scented forest earth . Snow piling up over the world, carrying liquid silver , and where the sun’s frame unfurls a chill ; dimming.

Table 1: Representative generated samples by iLLaDA-8B-Instruct ( Nie et al. (2026) ) with SoT . Coloring in the generated texts is generated by gemma-4-26B-A4B ( Gemma Team et al. (2026) ).

Model	Metric	none	end	middle	start
SEDD(medium) Lou et al. (2024)	GenPPL	$33.12_{\pm 0.58}$	$32.73_{\pm 0.57}$	$27.52_{\pm 0.43}$	$12.24_{\pm 0.26}$
	Entropy	$7.57_{\pm 0.02}$	$7.57_{\pm 0.02}$	$7.52_{\pm 0.03}$	$6.42_{\pm 0.05}$
	CoLA	$0.285_{\pm 0.006}$	$0.285_{\pm 0.006}$	$0.335_{\pm 0.006}$	$0.336_{\pm 0.007}$
	Qwen (%)	–	$51.5_{\pm 0.8}$	$\mathbf{62.0}_{\pm 1.3}$	$53.3_{\pm 1.9}$
	$\Delta R_{7}$ (pp)	$0.000_{\pm 0.000}$	$-0.005_{\pm 0.004}$	$0.025_{\pm 0.013}$	$1.405_{\pm 0.210}$
RADD(medium) Ou et al. (2025)	GenPPL	$33.66_{\pm 0.60}$	$33.34_{\pm 0.59}$	$28.08_{\pm 0.44}$	$13.71_{\pm 0.28}$

Table 2: Results at

\gamma=0.02

. Each condition contains 256 sequences for SEDD and RADD and 128 for iLLaDA. The no-remasking column is the shared control. All entries give mean

\pm

standard error; Qwen preference scores are expressed as percentages, and

\Delta R_{7}

in percentage points (pp). For

\Delta R_{7}

, the no-remasking baseline is zero by definition.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Case Name	Index	Prompt Content
story __forest_snow	$i=1$ (forest)	“Write a short story. In the forest, the tall trees are everywhere. Green leaves cover the ground and the air is warm. Small animals run through the grass. There are many plants and flowers in this place. The woods are full of life and nature.”
story __forest_snow	$i=2$ (snow)	“Write a short story. In the deep snow, everything is white and cold. The ground is covered in thick ice and the wind is freezing. No plants are growing here. It is a very quiet and winter world with high mountains of snow.”
key __forest_city	$i=1$ (forest)	“Write a short story about a traveler searching for a lost key. The story takes place in a dense forest. The traveler searches among trees, follows animal tracks, and hears birds overhead.”
key __forest_city	$i=2$ (city)	“Write a short story about a traveler searching for a lost key. The story takes place in a crowded city. The traveler searches among buildings, follows street signs, and hears traffic nearby.”
package __past_future	$i=1$ (past)	“Write a short story about a messenger delivering a mysterious package. The story takes place in a medieval kingdom. The messenger travels by horse, carries a sealed letter, and seeks a castle.”
package __past_future	$i=2$ (future)	“Write a short story about a messenger delivering a mysterious package. The story takes place in an interstellar civilization. The messenger travels by spacecraft, carries an encrypted message, and seeks a space station.”

Appendix

Table 3: Detailed prompt pairs used for SoP experiments.

Case Name	Index	Prompt Content
story_reasoning __forest_snow	$i=1$ (forest)	“ <chat> Write a short story. </chat> <think> The narrative must take place in a dense, vibrant summer forest. I need to emphasize overflowing life, humid warm air, lush green canopy, and the constant sounds of rustling leaves and active wildlife. The tone should feel alive, earthy, and teeming with natural energy. </think> ”
	$i=2$ (snow)	“ <chat> Write a short story. </chat> <think> The narrative must take place in a desolate, frozen winter tundra. I need to emphasize absolute stillness, sub-zero cold, blinding white snowstorms, and a landscape completely devoid of plant life. The tone should feel silent, harsh, and starkly isolated. </think> ”

Appendix

Table 4: Detailed prompt pairs used for SoT experiments.The specific tags, like <chat> and <think> , used are replaced depending on the model.

Generation condition	$N$	Forest reference	Snow reference
Forest only	512	$0.5367\pm 0.0047$	$0.3898\pm 0.0045$
Superposition, $\braket{\bm{\uppsi}_{0}\|\text{forest}}$	185	$0.5053\pm 0.0074$	$\mathbf{0.4477}\pm 0.0082$
Snow only	512	$0.3898\pm 0.0045$	$0.5439\pm 0.0047$
Superposition, $\braket{\bm{\uppsi}_{0}\|\text{snow}}$	327	$\mathbf{0.4294}\pm 0.0062$	$0.5252\pm 0.0063$

Appendix

Table 5: Average cosine similarity for the forest/snow experiment using MiniLM embeddings. Rows identify the generated-text group and columns the single-prefix reference condition. Superposition rows denote outputs from

\bm{\uppsi}_{t=0}

with the indicated component selected;

N

is the group size. Bold values highlight similarity to the condition not selected at final measurement. The sample counts and uncertainties are those reported for each group.

Figure 4: Component weight $p(i=1)$ for masked and uniform SEDD at two coefficient-update exponents. Rows show forest/city( key__forest_city in Table 3 ) and past/future( package__past_future in Table 3 ); each panel contains 120 trajectories. Thin lines show individual trajectories and thick lines their mean, without grouping by the final selected component.

Figure 5: Mean trajectory-level Shannon entropy for the same SEDD experiments with forest/city( key__forest_city in Table 3 ) and past/future( package__past_future in Table 3 ). Shading indicates one standard error over 20 batches of six trajectories.

Figure 6: Text-quality metrics at $\gamma=0.02$ . Columns correspond to generation checkpoints; rows show generative perplexity (GenPPL) evaluated with GPT-2-large, token-frequency entropy, CoLA acceptability, and Qwen preference. CoLA uses truncated classifier inputs (Appendix G.2 ). Higher Qwen preference favors remasking over no remasking. Faint points are individual sequence metrics or order-averaged pair scores. Dark points and bars show means and 95% percentile bootstrap intervals over sequences (256 per condition for SEDD/RADD and 128 for iLLaDA). GenPPL uses a logarithmic axis. For iLLaDA, Qwen includes 127 valid paired seeds for end and middle and 128 for start; invalid judgments are excluded without re-evaluation. The dashed line marks 50% preference; no self-comparison is evaluated for none .

Figure 7: Excess long-token-sequence repetition. Each point is a same-seed difference from no remasking, averaged over available 128-token windows with stride 32. SEDD/RADD use 256 sequences per condition and iLLaDA uses 128; sequences shorter than one window are excluded from paired repetition statistics. Bars are 95% bootstrap intervals over sequences, not windows. Values are in percentage points. All panels use the same vertical range,

-0.5

–6 percentage points, to resolve differences in the means. Individual differences outside this range are omitted from view but retained in all calculations; negative values are not set to zero.

Diffusion Language Models

Diffusion language models generate and refine text through iterative denoising, enabling bidirectional context and parallel token updates.

See all 473 papers

Unlocking Prompt Infilling Capability for Diffusion Language Models
Apr 4, 2026Yoshinari Fujinuma, Keisuke SakaguchiDiffusion Language Models Masked Diffusion Models
Patronus AI · Tohoku University, RIKEN
DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
May 22, 2026Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3Diffusion Language Models Diffusion Language Model Distillation
NVIDIA
How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
May 8, 2026Viacheslav Meshchaninov, Alexander Shabalin, Egor Chimbulatov +4Non-Autoregressive Text Generation Latent Diffusion Models
Constructor University, Bremen, Germany · HSE University, Moscow, Russia · Applied AI Institute, Moscow, Russia +2

Apr 4, 2026J/K move · Enter open · S save

Unlocking Prompt Infilling Capability for Diffusion Language Models

Yoshinari Fujinuma, Keisuke Sakaguchi

Patronus AI · Tohoku University, RIKEN

Diffusion Language Models Masked Diffusion Models Diffusion Language Model Training Diffusion Model Fine-Tuning Prompt Engineering Prompt Optimization

Open paper AskPDF arXiv

Masked diffusion language models (dLMs) generate text through bidirectional denoising, yet this capability remains locked for infilling prompts. This limitation is an artifact of the current supervised finetuning (SFT) convention of applying response-only masking. To unlock this capability, we extend full-sequence masking during SFT, where both prompts and responses are masked jointly. Once unlocked, the model infills masked portions of a prompt template conditioned on few-shot examples. We show that such model-infilled prompts match or surpass manually designed templates, transfer effectively across models, and are complementary to existing prompt optimization methods. Our results suggest that training practices, not architectural limitations, are the primary bottleneck preventing masked diffusion language models from infilling effective prompts

Abstract

Diffusion Language Models

Explore similar work

Unlocking Prompt Infilling Capability for Diffusion Language Models

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling

How to Train Your Latent Diffusion Language Model Jointly With the Latent Space