Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flow

Tung Sum Thomas Kwok; Yidong Ouyang; Yingjia Wan; Ying Nian Wu; Zhijiang Guo; Oscar Leong

cs.LGSep 20, 2026

Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flow

Authors: Tung Sum Thomas Kwok, Yidong Ouyang, Yingjia Wan, Ying Nian Wu, Zhijiang Guo, Oscar Leong

Organizations: University of California, Los Angeles · Shanghai Jiao Tong University

Abstract

Uniform discrete flow permits repeated updates at every generation position. While continued revision supports correction of wrong tokens, it also exposes correct intermediate predictions to later errors. An experiment on Sudoku puzzles shows that 9.4% of generated cells are correct at an intermediate step but incorrect in the final output. We introduce generation order into uniform discrete flow through selective absorption, which fixes chosen predictions while preserving the uniform flow velocity at active positions. To prioritize reliable predictions for absorption, we propose Low-Entropy Discrete Flow (LEDFlow), a training-free sampler that adaptively orders absorption by local entropy. By decomposing absorption error into joint dependence and conditional prediction terms, we show that, under entropy-error regularity, selecting the lowest-entropy positions under a fixed absorption count minimizes an upper bound on the conditional term. We further analyze sensitivity of global lookahead, whose worst-case decision-error bound grows with lookahead window under an imperfect denoiser. Across reasoning benchmarks, LEDFlow attains 0.845 Nikoli Sudoku solve accuracy, with largest gains on strongly constrained tasks. On a text-to-image generation benchmark it attains the best overall score among decode-time samplers, and on multimodal understanding it improves over the default sampler on all six benchmarks, at an inference cost comparable to standard flow sampling.

Figures & tables

Figure 1: Entropy-guided absorption in uniform discrete flow. (a) Late revisions can overwrite correct predictions. (b) Selective absorption fixes chosen predictions while other positions continue evolving. (c) LEDFlow prioritizes low-entropy positions for absorption; global lookahead aggregates potentially noisy estimates across the remaining positions.

Figure 2: Ordered absorption reduces late revisions and improves puzzle solving. (a,b) Token-change timing and cell accuracy on 100 Nikoli puzzles with

K=64

. Without absorption, cell accuracy falls rapidly from 0.58 to 0.43 at

t\approx 0.94

, whereas LEDFlow rises monotonically. (c) Statistics from the trajectory run: correct-to-wrong reversions decrease from 0.094 to 0.026 of all generated cells (cells correct at an intermediate step but incorrect at the endpoint).

Figure 3: Qualitative comparison across the structure spectrum. (a) Text-to-image on FUDOKI shows LEDFlow produces images with improved alignment with the required descriptions. (b) Sudoku: unordered Euler sampler repeats digits already present (two ‘1’s and ‘8’s in row 7 and 9), violating the row constraint, while LEDFlow conditions on absorbed cells and avoids the conflict.

sampler	POPE $\uparrow$	MME $\uparrow$	MMBench $\uparrow$	GQA $\uparrow$	MMMU $\uparrow$	MM-Vet $\uparrow$	Avg. rank $\downarrow$
Euler	86.1	1485.4	73.9	57.6	34.3	38.0	4.33
Euler (our replication)	87.4	1494.5	73.7	57.0	36.2	38.2	3.50
Exact guidance	86.8	1492.7	74.2	58.2	35.4	38.6	3.00
Top K-margin	83.5	1497.4	65.0	56.9	30.0	40.0	4.50
Info-Gain	89.3	1503.0	65.1	56.4	29.1	37.5	4.33
[2pt/1pt] LEDFlow (ours)	89.0	1504.6	74.6	58.0	37.4	40.5	1.33

Table 1: Multimodal understanding scores. White/grey/blue rows denote non-absorbing baselines/absorbing baselines/ LEDFlow . Bold , italic , and underlining mark first, second, and third place. Avg. rank is the mean benchmark rank (

\downarrow

). Higher scores are better (

\uparrow

). Entropy-guided selective absorption gives consistently higher but modest gains.

Figure 4: Empirical analysis of entropy-guided absorption and the score sensitivity of global lookahead. Red (Thm. 1 ): LEDFlow ’s entropy-guided absorption policy gives the highest puzzle accuracy (a), and its gain over each model’s default generation order transfers across pretrained models and tasks (b). Blue (Lemma 1 ): aggregating more lookahead terms increases score error and eventually reduces absorption accuracy (c), with increased inference runtime (d).

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Claim	Supported by	Boundary
Selective absorption separates the priority decision from the native flow update	Section 3.1 and Appendix C.1	Absorption deliberately changes the sampler law; it is not claimed to preserve the path law of the unmodified uniform flow.
The local absorption error separates into joint factorization and conditional prediction terms	Proposition 2 and Appendix C.3.1	The equality is local to a fixed state and selected block; errors across the complete sampling trajectory also depend on subsequent states.
Lowest predictive entropy minimizes the conditional-error upper bound	Theorem 1 and Appendix C.3	The result assumes entropy-error regularity on oracle-reachable states; it optimizes a local surrogate and does not cover trajectories after support is lost.
Global entropy reduction accumulates estimation error with lookahead window	Lemma 1 and Appendix E.7	This is an upper-bound and controlled fixed-state result; it does not imply that lookahead fails for every denoiser or state.
Entropy-guided absorption transfers across tasks and pretrained models	Section 4.2 and Appendix E.6	Transfer replaces each model’s default generation order through its own sampler; several differences are not significant. Sudoku runs share a checkpoint within each capacity; all rows of Table 3 share one matched cohort, while separate diagnostics retain their stated protocols.

Appendix

Table 4: Evidence and scope of the main claims.

Figure 5: The replacement rate is the product of residual error and hazard. (a) At

K{=}64

the per-step hazard

c_{k}

(red) rises to one as

t

approaches 1 while the incumbent error

\varepsilon_{t}

(grey) falls and then rises again in the last steps. Their product (blue), the correct-token replacement rate, is small for most of sampling and spikes at the endpoint. (b) The product across grids

K\in\{16,64,256\}

: the terminal spike grows with

K

, so refining the discretization increases the late churn.

Denoiser	$\kappa_{\theta}$	$\kappa^{\star}$	$B_{s}$	$C_{s}$	better policy
$6.4$ M	$0.527$	$0.698$	$0.0043$	$0.0100$	entropy
$21.3$ M	$0.695$	$0.397$	$0.0038$	$0.0025$	Info-Gain

Appendix

Table 5: Measured comparison on Sudoku-Extreme (600 paired reachable states per denoiser).

\kappa_{\theta}

is the probability that Info-Gain selects an oracle-consistent position and

\kappa^{\star}=C_{s}/(B_{s}+C_{s})

. The local entropy policy has no larger expected loss when

\kappa_{\theta}\leq\kappa^{\star}

Figure 6: Entropy-accuracy correlation, both denoisers. Same binned trajectory data as Table 7 (all 12 equal-count bins, 6.4M and 21.3M), plotted against the model’s predictive entropy

H_{\theta}

on a log axis. (a) Per-cell accuracy (

1{-}\mathrm{err}

) decreases with entropy at both capacities, with Spearman

\rho(H,\mathrm{err})=0.23

(6.4M) and 0.09 (21.3M) (the low

\rho

reflects that most mass sits in the flat, near-zero-entropy, near-ceiling regime common to reachable Sudoku states, and the drop is concentrated in the top bins, consistent with the nonlinear slope

L_{\phi}

reported below). (b) NLL of the true digit rises with entropy at both capacities. The 21.3M denoiser’s NLL curve sits uniformly below the 6.4M curve, the calibration improvement with capacity that motivates comparing the two models throughout this appendix.

Denoiser	Mean	95th percentile	99th percentile	Maximum
$6.4$ M	0.57	0.002	16.70	20.85
$21.3$ M	0.12	0.001	0.003	22.33

Appendix

Table 6: Offset

\phi_{0}

summaries for the relaxed envelope (nats). The mean column gives the offsets used to compute the reported coverage. The other columns describe the distribution’s upper tail.

$H_{\theta}$ (bin mean)	0.00	0.24	0.46	0.64	0.79	1.08
per-cell error	0.015	0.120	0.222	0.343	0.399	0.526
NLL (true digit)	0.27	0.65	0.88	1.12	1.01	1.25

Appendix

Table 7: Entropy-error diagnostic (Definition 1 ) on Nikoli (

n{=}100

): per-cell error and NLL of the true digit, binned by the model’s predictive entropy

H_{\theta}

over the whole sampling trajectory (6.4M denoiser, representative equal-count bins). Per-cell error rises monotonically with entropy. The bottom row reports the empirical error-entropy slope and surrogate error

\varepsilon

for both denoisers.

probe	regime (denoiser)	internal	external	gap
(a) density shift, test givens $26$ – $31$ , $n{=}185$
	calibrated (ECE $0.008$ )	0.973	0.746	$+0.227$
	overconfident (ECE $0.074$ )	0.827	0.546	$+0.281$
(b) wrong constraint: Latin-trained (box-blind), test Sudoku
	hard (givens $32$ – $40$ , floored)	0.109	0.124	$-0.015^{*}$
	medium (givens $40$ – $48$ )	0.637	0.630	$+0.007$

Appendix

Table 8: Two attempts to make a fixed external priority beat adaptive entropy-guided absorption by inducing miscalibration. internal is adaptive entropy and external is the best of the fixed external score and logical forcedness. Top: density shift (train dense, test sparse). Bottom: wrong constraint (train Latin, test Sudoku) over a difficulty sweep. The external order wins only in the floored hard band (a non-significant

{\sim}2

-puzzle margin). Wherever the denoiser is capable, confidence wins.

ID	Checkpoint / location	Decoder or absorption protocol	Cohort / sampling steps
M	Shared task-specific $6.4$ M; Table 3 , all rows; Fig. 2	Kinetic-optimal updates; absorbing rows add a cosine absorption schedule	Matched across all rows: same puzzles, Generated/Nikoli/Extreme $n=500/100/700$ , five seeds, $K=64$ ; Fig. 2 traces the Nikoli subset
A	Task-specific $6.4$ M; Table 24	Ordering-score ablation; cosine absorption schedule	Temperature sweep, seeds $0,1$ , $K=64$
L	Shared model at each capacity; Table 23	Singleton absorptions	Each set $n=100$ , five seeds
G	$6.4$ M and $21.3$ M; Table 20	Separate singleton-ordering study	Generated $n=1000$ , $22$ – $34$ givens

Appendix

Table 9: Protocol map for Sudoku results. All samplers at a given capacity share the same checkpoint. Each ID identifies a separate reported experiment, not an interchangeable estimate.

K

is the number of sampling steps. It need not equal the number of singleton absorptions. Solve accuracies and policy labels are those of Table 3 .

benchmark	$N$	native sampler	LEDFlow (ours)	metric	spread	$95\%$ CI (diff)
POPE	5127	87.4	89.0	acc	$+1.6$	$\pm 1.3$
MME	2374	1494.5	1504.6	P+R	$+10.1$	—
MMBench	4377	73.7	74.6	Overall	$+0.9$	$\pm 1.8$
GQA	12578	57.0	58.0	acc	$+1.0$	$\pm 1.2$
MM-Vet	218	38.2	40.5	GPT-judge	$+2.3$	$\pm 9.1$
MMMU	1050	36.2	37.4	acc	$+1.2$	$\pm 4.1$

Appendix

Table 10: Understanding suite on the full standard splits via FUDOKI’s VLMEvalKit. The table includes the six benchmarks of Table 3 . The last column is an approximate 95% interval on the native-sampler-vs- LEDFlow difference (

1.96\sqrt{2p(1-p)/N}

, evaluated at the native-sampler rate). Entropy-guided absorption improves every benchmark in the main-text suite, and the POPE margin clears the reported interval. The remaining margins are smaller than their intervals. This places short-answer understanding at the low-sensitivity end of the observed task spectrum, consistent with a smaller conditional term than on the constrained tasks in § 4.1 .

model	POPE	MME †	MMBench ‡	GQA	MMMU	MM-Vet §
Reported unified understanding-and-generation models (context)
LWM	75.2	—	—	44.8	—	9.6
Chameleon	—	—	—	—	22.4	8.3
Show-o-256	73.8	948.4	—	48.7	25.1	—
Show-o-512	80.0	1097.2	—	58.0	26.7	—
D-DiT	84.0	1124.7	—	59.2	—	—

Appendix

Table 11: Understanding suite in context: reported unified understanding-and-generation models alongside our frozen-FUDOKI decode-time samplers (condensed in Table 3 of the main text). Numbers are compiled from prior work [ Wan et al., 2026b ] . Understanding is a measurement-validity control, not a SOTA claim. Higher is better, “—” not reported.

method	Single	Two	Count	Colors	Pos.	Attr.	Overall $\uparrow$
Generation-only models (context)
SDXL [ Wan et al., 2026a ]	0.98	0.74	0.39	0.85	0.15	0.23	0.55
DALL-E 3 [ Wan et al., 2026a ]	0.96	0.87	0.47	0.83	0.43	0.45	0.67
SD3-Medium [ Wan et al., 2026b ]	0.99	0.94	0.72	0.89	0.33	0.60	0.74
FLUX.1-dev [ Wan et al., 2026a ]	0.98	0.93	0.75	0.93	0.68	0.65	0.82
Unified understanding-and-generation models (context)

Appendix

Table 12: GenEval in context: generation-only and unified understanding-and-generation models alongside the frozen-FUDOKI decode-time samplers of Table 3 (main text). The external rows modify or replace the model and are orthogonal to the decode-time comparison. External rows as compiled by their cited source. Higher is better, “—” not reported.

method	GenEval Overall
Frozen-model samplers (this setting)
native sampler (Euler@64, ours)	0.753
Euler@64 [ Wan et al., 2026c ]	0.754
time-/location-corrected [ Wan et al., 2026c ]	$0.765$
LEDFlow (ours)	0.781
Training-based (context)

Appendix

Table 13: GenEval Overall on FUDOKI in the masked discrete flow literature, separating decode-time samplers (frozen model, our setting) from training -based methods. Among frozen-model samplers the spread is

{\approx}0.05

, and an external FUDOKI Euler@64 result reproduces our native sampler to 0.001. LEDFlow is the strongest decode-time entry in this comparison, although its margin over the corrected-sampler family is within the benchmark’s resolution. Training-based methods modify the model and address a separate axis.

Measurement	Euler	LEDFlow	Diff. [ $95\%$ CI]
Token changes per position	$4.14$	$2.33$	$+1.81$ [ $1.72,1.88$ ]
excluding the final step	$3.44$	$2.30$	$+1.13$ [ $1.04,1.21$ ]
Positions changed more than once	$0.97$	$0.73$	$+0.24$ [ $0.22,0.25$ ]
Changes occurring after $t{=}0.75$	$0.45$	$0.11$	$+0.34$ [ $0.34,0.35$ ]
Mean time of last change	$0.96$	$0.54$	$+0.42$ [ $0.41,0.43$ ]
GenEval accuracy	$0.753$	$0.781$	$-0.028$ [ $-0.037,-0.020$ ]

Appendix

Table 14: Text-to-image trajectory measurements. FUDOKI text-to-image (GenEval prompts,

K{=}64

). Token statistics are per image token, with 95% bootstrap intervals over images for the difference, and correctness flips are the fraction of samples whose GenEval verdict changes between

t{=}0.75

and the endpoint, on in-distribution GenEval prompts and on the 16 counterfactual out-of-distribution prompts of Table 15 . The Sudoku counterpart of these measurements is Figure 2 (c). ⋆ GenEval prompts are in-distribution for the pretrained backbone, so late revisions there can recover memorized configurations. The counterfactual prompts isolate the same measurement without that confound, and under them correct-to-wrong exceeds wrong-to-correct.

Figure 7: FUDOKI text-to-image trajectories, native uniform flow versus LEDFlow . (a,b) Image-token change density over sampling time: the native flow accumulates most of its changes after

t{=}0.75

, while LEDFlow front-loads absorption and its density decays. (c) GenEval accuracy of the image decoded from the intermediate state, on in-distribution GenEval prompts (solid) and on the 16 counterfactual prompts of Table 15 (dashed). LEDFlow reaches its final quality by

t{\approx}0.6

and holds it. The native flow reaches comparable quality only at the endpoint, after the late burst of changes in (a,b).

Tag	Prompts
Colors	a photo of a green cow; a photo of a purple giraffe; a photo of a blue banana; a photo of a pink elephant
Counting	a photo of eight apples; a photo of six cats; a photo of seven birds; a photo of five clocks
Two objects	a photo of a toothbrush and a giraffe; a photo of a pizza and a fire hydrant; a photo of a laptop and a cow; a photo of an elephant and a spoon
Position	a photo of a car above a bird; a photo of a boat above a bed; a photo of a bicycle above an elephant; a photo of a cup above a train

Appendix

Table 15: Out-of-distribution prompts for the text-to-image trajectory analysis. Sixteen counterfactual prompts, four per GenEval tag, written so that the requested attribute, count, pairing, or spatial relation is unlikely under the training distribution. They are scored with the standard GenEval detector pipeline and are the dashed curves in Figure 7 and the OOD rows of Table 14 .

Arm	Cell acc.	Solve	Correct $\to$ wrong	Changes/cell	Late-change frac.	Bad-absorption
Uniform flow (no absorption)	$0.413$	$0.000$	$0.281$	$1.87$	$0.60$	—
[2pt/1pt] + arbitrary absorption order	$0.723$	$0.410$	$0.000$	$1.00$	$0.37$	$0.280$
+ entropy order ( LEDFlow )	$\mathbf{0.935}$	$\mathbf{0.845}$	$0.000$	$1.00$	$0.37$	$\mathbf{0.066}$
+ entropy order, sampled absorption	$0.923$	$0.821$	$0.000$	$1.00$	$0.37$	$0.078$

Appendix

Table 16: Absorption-policy diagnostic on Nikoli (

n{=}100

, 6.4M denoiser,

K{=}64

, seeds 0 to 4). Bad-absorption is the fraction of absorption events whose absorbed value is wrong. The arbitrary-order and entropy-order solve accuracies are the values of Table 3 . The no-absorption arm never fixes a position, so no puzzle completes. Paired puzzle-level bootstrap intervals for the cell-accuracy differences: no absorption to arbitrary +0.310

[0.285,0.336]

. Arbitrary to entropy +0.212

[0.166,0.256]

Sampler	Generated	Nikoli	Sudoku-Extreme
Euler	0.610	0.410	0.078
FreeCorrection	0.630	0.400	0.126
Time-corrected	0.610	0.410	0.078
Location-corrected	0.588	0.360	0.096
Top K-margin	0.820	0.796	0.271
Info-Gain	0.804	0.780	0.231

Appendix

Table 17: Sudoku solve accuracy by sampler. These values and policy labels reproduce Table 3 . All rows share the same checkpoint, puzzles, seeds, and

K{=}64

(protocol M). The identical Euler and Time-corrected rows are a discretization effect, explained in Appendix E.5.4 .

	Generated		Nikoli		Sudoku-Extreme
Entropy at absorption	err	$n$	err	$n$	err	$n$
$[0.00,0.05)$	$0.059$	$26496$	$0.086$	$5540$	$0.448$	$36240$
$[0.05,0.10)$	$0.700$	$20$	$0.167$	$6$	$0.550$	$866$
$[0.10,0.20)$	$0.750$	$16$	$0.333$	$6$	$0.570$	$732$
$[0.20,0.40)$	$0.750$	$16$	$0.778$	$9$	$0.537$	$555$
$[0.40,0.80)$	$0.600$	$35$	$0.800$	$10$	$0.622$	$585$

Appendix

Table 18: Absorption error at absorbed positions ( LEDFlow gate, seed 0, flow-active decode): empirical error rate binned by entropy at absorption, with the number of absorbed positions per bin. Low entropy predicts low error on generated/Nikoli. The relation fails under hard-puzzle over-confidence (Sudoku-Extreme). “—” marks empty bins.

Sampler	Generated	Nikoli	Extreme	Latin	Graph	Molec.
Uniformization	0.562	0.240	0.074	0.694	1.000	0.831
Fixed-score absorption	0.702	0.420	0.158	0.946	1.000	—

Appendix

Table 19: Structured-task results for uniformization and fixed-score absorption. Higher is better. “—” denotes the SMILES setting without a static relevance score.

			vs Euler	by #givens (determined)
absorption policy	solve-acc	$95\%$ CI	(McNemar)	$22$ – $25$	$26$ – $29$	$30$ – $34$
entropy guided (adaptive)	0.922	$[.905,.938]$	$+.486$ , $p{<}10^{-12}$	0.837	0.949	0.967
logical-forcedness (MRV)	0.621	$[.591,.651]$	$+.185$ , $p{<}10^{-12}$	—	—	—
fixed external score	0.527	$[.496,.558]$	$+.091$ , $p{=}9{\times}10^{-7}$	0.288	0.493	0.775
arbitrary-order absorption	0.446	$[.415,.477]$	—	—	—	—
Euler@64	0.436	$[.406,.467]$	—	0.190	0.358	0.740

Appendix

Table 20: Sudoku at scale (

n{=}1000

generated puzzles, 6.4M, #givens

\in[22,34]

): solve accuracy with 95% puzzle-level bootstrap CIs and paired McNemar tests. Upper block: paired against Euler. The adaptive and fixed-score absorption policies are both significant, and logical forcedness underperforms the model’s entropy policy (an imperfect denoiser tracks calibration, not forcedness). Lower block: the competing single-position ordering policies on the same 1000 puzzles, paired against the entropy policy. The three confidence-based scores (entropy, probability margin, maximum probability) are statistically tied, whereas Info-Gain (window 8) is significantly below them and unordered Euler far below. Wall-clock per puzzle on one H100 is 6.9 ms (entropy), 6.2 (margin), 2.4 (max-probability), 54.5 (Info-Gain), and 8.6 (Euler, re-run at 0.427, within the interval of the upper-block value). On the 21.3M denoiser the entropy policy remains ahead of Info-Gain (0.964 against 0.947,

p{=}0.027

) at 9.5 against 149.3 ms. Right block: accuracy stratified by difficulty, where the ordering gap grows as puzzles harden. We note that this is not Theorem 1 (ii) directly, which concerns the limit in which the denoiser approaches the oracle. The two agree only under the additional premise that harder puzzles leave the denoiser further from the oracle, which is plausible here but is not established by the theorem.

absorption priority	argmax value	sampled value
entropy guided	$\mathbf{0.845}$	$0.874\pm 0.008$
arbitrary order (uniform choice, Zheng et al. [2025] )	$0.380\pm 0.031$	$0.280\pm 0.026$
Euler@64 (non-absorbing flow, argmax): $0.410$

Appendix

Table 21: Decision-rule control (Nikoli, 6.4M,

n{=}100

, 5 seeds): absorption priority crossed with value selection , all single-position with per-step reevaluation. Priority dominates at both absorption rules. The argmax-vs-sample rule is a minor effect. The unordered + argmax cell is the arbitrary-order policy follows the uniform first-hitting choice of Zheng et al. [2025] . The entropy-guided and Euler entries are the Table 3 values for this cohort. Puzzle-level CIs and paired McNemar tests are in Table 20 .

	Nikoli $\uparrow$		Sudoku-Extreme $\uparrow$
Sampler	6.4M	21.3M	6.4M	21.3M
Uniformization	0.240	0.630	0.074	0.194
Euler	0.410	0.820	0.078	0.224
Fixed external score	0.420	0.870	0.158	0.312
FreeCorrection	0.400	0.780	0.126	0.270
Time-corrected	0.410	0.820	0.078	0.227

Appendix

Table 22: Sensitivity to Sudoku denoiser capacity (solve accuracy). Parameter counts are exact up to one decimal place. The 6.4M entries of the LEDFlow and Euler rows are the values of Table 3 . The models also differ in training steps and data mixture, so the comparison demonstrates robustness across denoisers rather than an isolated size effect. The middle block isolates absorption priority by holding the denoiser and argmax value rule fixed. Accuracy decreases monotonically from entropy-guided to arbitrary and then high-entropy-first absorption.

Absorption order	Nikoli $\uparrow$		Sudoku-Extreme $\uparrow$		Generated $\uparrow$
Absorption order	$6.4$ M	$21.3$ M	$6.4$ M	$21.3$ M	$6.4$ M	$21.3$ M
Low-entropy-first (ours)	0.845	0.970	0.269	0.514	0.865	0.996
Offline $\sigma_{\mathrm{rel}}$ (deployable)	0.420	0.870	0.143	0.301	0.840	0.960
Euler @ $64$ (flow)	0.410	0.820	0.078	0.224	0.610	0.924
Arbitrary single-position	0.340	0.820	0.104	0.256	0.756	0.948
High-entropy-first (reversal)	0.110	0.550	0.051	0.117	0.580	0.860

Appendix

Table 23: The absorption-order ladder. Solve accuracy for adaptive low-entropy, offline-score, arbitrary, and high-entropy orders. Absorbing policies share the denoiser and argmax rule within each capacity; the non-absorbing Euler row is a separate reference. These results test position ordering under the uniform discrete flow. The larger spread for the weaker denoiser is an empirical observation, not a prediction of Theorem 1 without further assumptions.

$T$	Generated	Nikoli	Sudoku-Extreme
$0.5$	$+0.005$	$-0.010$	$\mathbf{+0.007}$
$0.7$	$+0.005$	$-0.010$	$+0.005$
$1.0$ (default)	$0.000$	$0.000$	$0.000$
$1.5$	$\mathbf{+0.010}$	$-0.020$	$-0.011$
$2.0$	$+0.005$	$-0.010$	$+0.002$

Appendix

Table 24: Temperature scaling of the ordering entropy (6.4M denoiser,

K{=}64

T

rescales the logits used for the ordering score only. The absorbed value is the argmax of the unscaled posterior. Entries are changes in solve accuracy relative to the default

T{=}1

of Table 3 , measured within one sweep, so no entry restates an absolute accuracy for a configuration reported elsewhere.

Model	Benchmark	Backbone family	Default order	LEDFlow order	$\Delta$
Text reasoning (native sampler, order-only ablation)
LLaDA-8B-Instruct [ Nie et al., 2025 ]	GSM8K	masked diffusion LM	0.725	0.770	$+0.045$
Dream-7B [ Ye et al., 2025b ]	GSM8K	masked diff. LM (uniform prior)	0.371	0.492	$+0.121$ ( $z{=}5.5$ , $p{\approx}5{\times}10^{-8}$ )
MMaDA-8B [ Yang et al., 2025 ]	GSM8K	unified MM diffusion	0.246	0.474	$+0.228$ ( $p{\approx}2{\times}10^{-18}$ )
MMaDA-8B	BBH logical deduction	unified MM diffusion	0.370	0.373	$+0.003$ ( $\chi^{2}{=}0.00$ , n.s.)
MMaDA-8B	Tracking shuffled objects	unified MM diffusion	0.180	0.240	$+0.060$ ( $\chi^{2}{=}3.11$ , $p{\approx}0.08$ )

Appendix

Table 25: Full transfer results across pretrained diffusion-model families. We vary only the absorption policy through each model’s native sampler. The text-reasoning experiments use matched wall-clock time. Higher is better. The label n.s. denotes a difference that is not statistically significant. Every cell shown in Figure 4 (b) appears here. The cells left grey in that figure are not applicable rather than pending: LLaDA-8B and Dream-7B are text-only diffusion language models and are not evaluated on the image-conditioned benchmarks, while LLaDA-V is an understanding-only multimodal model without a generation head and is therefore evaluated on multimodal mathematics alone.

sampler	denoiser evals.	ms/puzzle	solve-acc
LEDFlow (single-position absorption)	55.7	10.5	$\mathbf{0.845}$
Euler@16	16	3.6	$0.238\pm 0.014$
Euler@32	32	5.7	$0.290\pm 0.038$
Euler@64	64	9.6	$0.410$
Euler@128	128	17.5	$0.334\pm 0.022$
Euler@256	256	33.3	$0.350\pm 0.030$

Appendix

Table 26: Sudoku efficiency (6.4M denoiser, Nikoli

n{=}100

, mean 55.7 blanks/puzzle, five seeds, one H100). The single-position LEDFlow policy evaluates the denoiser once per absorbed position. Solve accuracies for LEDFlow and Euler@64 are the Table 3 values. Table 20 reports puzzle-level bootstrap intervals and paired McNemar tests.

	$6.4$ M		$21.3$ M
sampler	acc	ms	acc	ms
LEDFlow (single-position absorption)	0.925	5.7	0.970	9.7
Euler@64 / @256	0.520/0.560	8.2/28.2	0.830/0.810	12.8/46.1
uniformization (exact, unord.)	0.425	7.3	0.750	11.3

Appendix

Table 27: Profiled wall-clock (generated puzzles,

n{=}200

, mean

{\sim}52

blanks, one H100, and ms/puzzle includes each policy’s score computation and denoiser reevaluation). LEDFlow ’s accuracy advantage over the non-absorbing baselines grows with denoiser quality (to 0.970 at 21.3M), while uniformization remains less accurate at comparable wall-clock time.

Block size	Nikoli ( $n{=}100$ )		Generated ( $n{=}500$ )
	solve / cell / bad	NFE	solve / cell / bad	NFE
Single ( $b{=}1$ )	$0.900$ / $0.936$ / $0.065$	$58$	$0.928$ / $0.969$ / $0.033$	$59$
[2pt/1pt] Fixed $b{=}2$	$0.880$ / $0.922$ / $0.079$	$29$	$0.918$ / $0.966$ / $0.036$	$30$
Fixed $b{=}4$	$0.850$ / $0.918$ / $0.083$	$15$	$0.892$ / $0.959$ / $0.043$	$15$
Fixed $b{=}8$	$0.720$ / $0.875$ / $0.127$	$8$	$0.830$ / $0.941$ / $0.061$	$8$

Appendix

Table 28: Block size (Sudoku, 6.4M denoiser, entropy ordering). Each cell reports puzzle solve accuracy / cell accuracy / fraction of incorrect absorptions, with the number of denoiser evaluations per puzzle. Larger blocks trade accuracy for fewer evaluations.

Givens	Multi-solution	Single ( $b{=}1$ )	Block $b{=}4$	Block $b{=}8$	Unique, any $b$
$22$	$150/150$	$1.000$	$0.507$	$0.267$	—
$26$	$150/150$	$1.000$	$0.667$	$0.207$	—
$30$	$147/150$	$0.993$	$0.708$	$0.460$	$1.000$

Appendix

Table 29: The joint term isolated (Sudoku, 6.4M denoiser, entropy ordering,

n{=}150

puzzles per givens level). Fraction of valid final grids on multi-solution puzzles as a function of block size. The last column reports the unique-solution puzzles in the same sweep.

Sampler	$\epsilon$	Nikoli ( $n{=}100$ )		Generated ( $n{=}500$ )
		solve / cell / bad	NFE	solve / cell / bad	NFE
LEDFlow (single)	$0$	$0.900$ / $0.936$ / $0.065$	$55.7$	$0.928$ / $0.969$ / $0.033$	$53.2$
[2pt/1pt] EB-Sampler	$0.5$	$0.760$ / $0.898$ / $0.104$	$4.2$	$0.792$ / $0.932$ / $0.070$	$3.7$
EB-Sampler	$1.0$	$0.670$ / $0.871$ / $0.130$	$3.6$	$0.664$ / $0.903$ / $0.100$	$3.2$
EB-Sampler	$2.0$	$0.520$ / $0.829$ / $0.173$	$3.0$	$0.528$ / $0.879$ / $0.124$	$2.7$
EB-Sampler	$4.0$	$0.270$ / $0.791$ / $0.211$	$2.6$	$0.334$ / $0.846$ / $0.158$	$2.3$

Appendix

Table 30: EB-Sampler reimplementation on Sudoku (6.4M denoiser). Cells report solve accuracy / cell accuracy / fraction of incorrect absorptions and the number of denoiser evaluations per puzzle.

\epsilon

is the cumulative-entropy threshold in nats.

\epsilon{=}0

is LEDFlow .

Flow-Based Generative Modeling

Flow-based generative modeling learns probability transports from simple noise to data for fast sampling, density estimation, and controllable generation.

See all 555 papers

Jun 28, 2026

Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasoners

Structured reasoning requires making and revising interdependent decisions to reach a globally consistent solution. Existing architectures struggle with this: autoregressive models commit sequentially and cannot revise earlier decisions, while masked diffusion models often require careful decoding schemes to coordinate interdependent predictions. We introduce Flow Reasoning Models (FRMs), a novel framework for structured reasoning that adapts continuous flows over discrete structured outputs with a simple recurrent refinement mechanism. By self-conditioning a flow model on its own past outputs, we turn one-shot denoising into iterative solution refinement. This lets FRMs make and revise decisions in parallel, efficiently coordinating interdependent choices across solutions. Yet conventional self-conditioning becomes unreliable at greater recurrent depth due to exposure bias between one-step training predictions and recursively generated inference states. We address this mismatch with Fixed-Point Forcing (FPF), which trains FRMs on states produced by their own inference dynamics while preserving the standard flow-matching objective. FRMs achieve solve rates of

99.5\%

100.0\%

, and

99.9\%

on Sudoku-Extreme, Zebra, and Maze-Unique, respectively. On Sudoku-Extreme, FRMs achieve higher peak accuracy than the evaluated masked-diffusion and specialized reasoning baselines while remaining highly compute-efficient, matching the next-best method's

98.7\%

peak solve rate with

44\times

fewer inference FLOPs.

Alec Helbling, Andrey Bryutkin, Mauro Martino +3

1Georgia Tech · 3MIT-IBM Computing Research Lab · 4IBM Research +1

Self-Conditioning Structured Reasoning Flow-Based Generative Modeling Constraint Satisfaction Iterative Refinement Algorithmic Reasoning+1

Ask PDF

Apr 6, 2026

Training-Free Refinement of Flow Matching with Divergence-based Sampling

Flow-based models learn a target distribution by modeling a marginal velocity field, defined as the average of sample-wise velocities connecting each sample from a simple prior to the target data. When sample-wise velocities conflict at the same intermediate state, however, this averaged velocity can misguide samples toward low-density regions, degrading generation quality. To address this issue, we propose the Flow Divergence Sampler (FDS), a training-free framework that refines intermediate states before each solver step. Our key finding reveals that the severity of this misguidance is quantified by the divergence of the marginal velocity field that is readily computable during inference with a well-optimized model. FDS exploits this signal to steer states toward less ambiguous regions. As a plug-and-play framework compatible with standard solvers and off-the-shelf flow backbones, FDS consistently improves fidelity across various generation tasks including text-to-image synthesis, and inverse problems.

Yeonwoo Cha, Jaehoon Yoo, Semin Kim +3

KAIST

Flow Matching T2I Generation Flow-Based Generative Modeling

Ask PDF

May 15, 2026

Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers

For a fixed flow-based generative model under a small inference budget, sample quality can depend strongly on where the sampler spends its few function evaluations. Flow matching and Schrödinger bridges define probability paths, yet their inference grids are usually heuristic or inherited from one-endpoint diffusion. We derive a conditional-marginal entropy-rate objective for bridge-aware discretization, separating endpoint-conditioned bridge geometry from marginal flow evolution, and use it to build a training-free entropic inference-time scheduler from first principles. For Gaussian Brownian bridges this rate is closed-form and U-shaped, motivating boundary-heavy nonuniform grids. On trained two-dimensional bridge/flow models, the estimated profile recovers the predicted shape and improves 10-step ODE-Heun MMD over linear by 18.1%, with a paired 22.7% SDE-Heun improvement in the same low-NFE sweep. On EDM/CIFAR-10, the entropic time-discretization gives the best tested five-step FID (186.3 \pm 4.0 versus 200.5 \pm 2.9 for linear and 238.0 \pm 5.3 for cosine). On AlphaFlow protein generation, entropic conditional-marginal (cond-marg) scheduling shows advantage in low-NFE regimes on both CAMEO22 and ATLAS benchmarks. These results support entropy-rate scheduling as a practical low-budget allocation signal for high-dimensional bridge and flow samplers.

Bruno Trentini, Dejan Stancevic, Michael M. Bronstein +2

NVIDIA Corporation, Santa Clara, CA, USA · University of Oxford, Dept. of Computer Science, Oxford, UK · Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, NL +1

Flow Matching Schrödinger Bridges Few-Step Diffusion Sampling Adaptive Sampling Flow-Based Generative Modeling Diffusion Model Inference Acceleration

Ask PDF

Abstract

Flow-Based Generative Modeling

Explore similar work

Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasoners

Training-Free Refinement of Flow Matching with Divergence-based Sampling

Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers