Dancing Diffusion: Learning Continuous Programs for Image Generation and Editing

Cui Enhe; Jinyue Yang; Bojian Yin; Guoqi Li

cs.AIOct 20, 2026

Dancing Diffusion: Learning Continuous Programs for Image Generation and Editing

Authors: Cui Enhe, Jinyue Yang, Bojian Yin, Guoqi Li

Organizations: Renmin University of China · Institute of Automation, Chinese Academy of Sciences

Abstract

Diffusion models generate samples through a multi-step denoising trajectory, yet condition every step on the same instruction, so the trajectory cannot be addressed at the level of individual operations. We introduce Dancing Diffusion, which assigns each span of timesteps a dedicated continuous condition block and thereby turns conditioning into a latent program. A compiler maps text features into this program, and a fixed pretrained generator executes its blocks over successive sampling intervals. The trajectory can then dance, keeping its executed history while its continuation is redirected. A shared prefix is executed once and reused, and a later operation is rewritten while content fixed by earlier operations is preserved. The compiler is trained through the images it produces, with visual feedback on both the requested change and the retained content. On SANA, reusing an executed prefix preserves markedly more of the scene than regenerating from noise at a similar strength of edit, learned conditions preserve more background than native prompt switching from the same saved state, and the learned program improves generation quality over native SANA. Later branches weaken the edit, which leaves semantic control over individual operations limited. By converting a single-instruction trajectory into a program of addressable operations, Dancing Diffusion lays a foundation for structured intermediate computation in diffusion models.

Figures & tables

Figure 1: Dancing: editing the generation path. (a) A background-only first block produces a common parent, shown as a predicted-clean readout. Both continuations reuse the same saved noisy state and introduce a sailboat or yacht through future conditions. A native same-noise generation is shown for orientation. Curves and intervals are schematic; the images illustrate the operation and are not a final-checkpoint comparison or evidence of native-method superiority. (b) A compiler produces conditions for a shared generator, with training-only feedback from a visual observer.

Figure 2: Training the conditions used after a saved prefix. The reference path supplies a detached sampler state and source image. The target compiler provides future conditions; the suffix output receives observer feedback for change and retention. Only compiler parameters are optimized. The source reference cannot move to make the edit loss easier. At deployment the observer and loss are absent.

Figure 3: Learning when to reconstruct foreground and background. (a) Exact-mask subject and background MSE on 1,024 validation images, normalized pointwise by the single-expert MSE, so that one denotes parity and lower is better. Every measured point is retained, and absolute curves are in Appendix F . (b) Predicted-clean $\hat{x}_{0}$ readouts of separate pure-noise DDPM-200 trajectories with common initial and per-step noise, followed by the target condition image. Row colors match the curves in (a). The observer-guided model delays wall texture while keeping a recognizable sphere. These trajectories are not samples from the noised-reference evaluation in (a), and the example was selected post hoc for correct colors, so it does not estimate how often colors are correct.

Method	3/9/18	6/12/12	9/12/9
Full regeneration	0.383 / 13 / 27	0.334 / 11 / 29	0.307 / 14 / 29
Learned suffix	0.254 / 14 / 29	0.247 / 15 / 27	0.242 / 14 / 25
Native suffix	0.269 / 14 / 28	0.274 / 14 / 26	0.269 / 17 / 27
SDEdit–SANA	0.284 / 16 / 28	0.288 / 16 / 27	0.293 / 12 / 27
FlowEdit–SANA	0.143 / 13 / 27	0.138 / 12 / 25	0.142 / 12 / 24

Table 1: Matched category-cohort comparison: 12 families, three seeds, 36 attempts per schedule. Cells give BG LPIPS

\downarrow

/ DINO replacement count

\uparrow

/ OWLv2 replacement count

\uparrow

. Mask coverage is 32/36, 33/36 and 33/36; counts use all attempts. Regeneration starts at 0; others at 9. Native uses the saved state; SANA-adapted image editors use the source PNG. Counts are proxies, not full-caption success; all starts appear in Appendix E .

Panel of Figure 4 — Figure 4: Editing with visible scene retention. Genuine, uncropped outputs for the same source and request: final checkpoint, schedule 6/12/12, branch/start 9 (regeneration: 0). The learned suffix retains avenue and building structure; image-input SANA adaptations also retain substantial context. Their source access differs (saved state versus PNG). These selected illustrations do not estimate success rates; failure cases are described in Appendix P .

Method / schedule	Initialization	Update 600	Final 1423	$\Delta$ vs. native
Native SANA	7.349 (no adaptation)			—
3/9/18	7.248	7.203	7.315	-0.034
6/12/12	7.268	7.247	7.145	-0.204
9/12/9	7.411	7.466	7.375	+0.026

Table 2: Generation FID versus native SANA and across training stages. Negative final-minus-native differences indicate improvement. All entries use the same noise set; no training-seed uncertainty is inferred.

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Source access	Question
Full regeneration	Same initial noise	Value of retaining source history
Native suffix	Same saved state	Value of learned future conditions
Initialized compiler	Same saved state	Effect of adaptation
SDEdit–SANA adaptation	Source PNG, re-noising	Image-input denoising control
FlowEdit–SANA adaptation	Source PNG, source/target velocities	Image-input flow editing control

Appendix

Table 3: Roles of the comparison methods.

Detector	Threshold	Audit precision	Audit recall
Grounding DINO tiny	0.40	0.886	0.819
OWLv2 base ensemble	0.31	0.900	0.861

Appendix

Table 4: Independent detector calibration on public images, not generated-editing accuracy.

Figure 5: Preservation and category change on a common evaluation cohort. Each point uses the same 12 category families and three seeds. Replacement proxies use all 36 attempts, and background LPIPS is defined for 32, 33 and 33 source masks across the schedules. Colors identify methods, marker shapes identify starts 9, 15 and 21, and the star denotes full regeneration. Curves connect descriptive means, not fitted frontiers, and the two detectors can order methods differently.

Method	Start	BG LPIPS $\downarrow$	Coverage	DINO T/R	OWLv2 T/R
Full regeneration	0	0.383	32/36	35/13	36/27
Learned suffix	9	0.254	32/36	36/14	36/29
Learned suffix	15	0.125	32/36	34/12	35/25
Learned suffix	21	0.037	32/36	30/0	17/8
Native suffix	9	0.269	32/36	36/14	36/28
Native suffix	15	0.122	32/36	34/13	35/23

Appendix

Table 5: Category proxies and preservation, schedule 3/9/18. T/R are counts out of 36 per detector; coverage is defined/36. Start is a saved-state branch for suffix methods and a re-noising or flow start for image-input methods.

Method	Start	BG LPIPS $\downarrow$	Coverage	DINO T/R	OWLv2 T/R
Full regeneration	0	0.334	33/36	34/11	35/29
Learned suffix	9	0.247	33/36	35/15	36/27
Learned suffix	15	0.129	33/36	34/13	33/26
Learned suffix	21	0.038	33/36	31/1	23/11
Native suffix	9	0.274	33/36	35/14	36/26
Native suffix	15	0.131	33/36	35/13	33/24

Appendix

Table 6: As Table 5 , schedule 6/12/12.

Method	Start	BG LPIPS $\downarrow$	Coverage	DINO T/R	OWLv2 T/R
Full regeneration	0	0.307	33/36	35/14	35/29
Learned suffix	9	0.242	33/36	35/14	34/25
Learned suffix	15	0.121	33/36	35/13	34/25
Learned suffix	21	0.043	33/36	30/2	21/6
Native suffix	9	0.269	33/36	36/17	36/27
Native suffix	15	0.126	33/36	36/11	34/23

Appendix

Table 7: As Table 5 , schedule 9/12/9.

Schedule	Method	Start 9	Start 15	Start 21	Coverage
3/9/18	SDEdit–SANA	0.265	0.288	0.269	63/72
3/9/18	FlowEdit–SANA	0.122	0.066	0.041	63/72
6/12/12	SDEdit–SANA	0.273	0.291	0.273	61/72
6/12/12	FlowEdit–SANA	0.122	0.067	0.042	61/72
9/12/9	SDEdit–SANA	0.282	0.296	0.282	60/72
9/12/9	FlowEdit–SANA	0.127	0.066	0.042	60/72

Appendix

Table 8: Background LPIPS of image-input SANA adaptations over all 72 source cases per schedule, using the historical source masks.

Figure 6: Absolute subject and background MSE on logarithmic scales, before normalization, with the fuller eight-step qualitative trajectory. The quantitative noised-reference evaluation and the selected pure-noise trajectory are separate experiments.

Schedule; branch	CLIP $\Delta$	BG LPIPS $\downarrow$	BG PSNR $\uparrow$	BG SSIM $\uparrow$
3/9/18; 0	0.097 (72/72)	0.352 (63/72)	15.652 (63/72)	0.572 (63/72)
3/9/18; 9	0.092 (72/72)	0.225 (63/72)	19.073 (63/72)	0.701 (63/72)
3/9/18; 15	0.076 (72/72)	0.108 (63/72)	24.316 (63/72)	0.837 (63/72)
3/9/18; 21	0.028 (72/72)	0.030 (63/72)	31.710 (63/72)	0.946 (63/72)
6/12/12; 0	0.092 (72/72)	0.321 (61/72)	16.500 (61/72)	0.599 (61/72)
6/12/12; 9	0.088 (72/72)	0.227 (61/72)	19.096 (61/72)	0.697 (61/72)

Appendix

Table 9: Final edit-only 24-family diagnostics: means (defined/attempted cases). Three seeds per request; step 0 is full regeneration. PSNR with zero MSE is undefined numerically and excluded from its mean.

Schedule; branch	CLIP $\Delta$	BG LPIPS $\downarrow$	BG PSNR $\uparrow$	BG SSIM $\uparrow$
3/9/18; 0	0.049 (72/72)	0.415 (10/72)	15.553 (10/72)	0.561 (10/72)
3/9/18; 9	0.042 (72/72)	0.198 (10/72)	21.826 (10/72)	0.780 (10/72)
6/12/12; 0	0.048 (72/72)	0.451 (11/72)	15.372 (11/72)	0.538 (11/72)
6/12/12; 9	0.041 (72/72)	0.244 (11/72)	20.267 (11/72)	0.729 (11/72)
9/12/9; 0	0.047 (72/72)	0.425 (12/72)	16.233 (12/72)	0.598 (12/72)
9/12/9; 9	0.042 (72/72)	0.252 (12/72)	20.327 (12/72)	0.745 (12/72)

Appendix

Table 10: As Table 9 , for the 24-case challenge pool.

Schedule; branch	Whole LPIPS	PNG MSE	Latent MSE
3/9/18; 9	0.050 (24/24)	0.003 (24/24)	0.020 (24/24)
3/9/18; 15	0.018 (24/24)	0.001 (24/24)	0.004 (24/24)
3/9/18; 21	0.005 (24/24)	0.000 (24/24)	0.000 (24/24)
6/12/12; 9	0.057 (24/24)	0.003 (24/24)	0.019 (24/24)
6/12/12; 15	0.031 (24/24)	0.001 (24/24)	0.006 (24/24)
6/12/12; 21	0.012 (24/24)	0.000 (24/24)	0.001 (24/24)

Appendix

Table 11: Same-prompt drift at the final checkpoint: eight evaluation families, three seeds. No target edit is requested; these distances measure change from the frozen-source continuation.

Branch	Control	IF	VC	VQ
9	Initial	11.1/6.9/22.2/59.7/0.0	9.7/12.5/34.7/43.1/0.0	9.7/22.2/41.7/26.4/0.0
9	Update 600	9.7/9.7/11.1/69.4/0.0	8.3/8.3/25.0/58.3/0.0	11.1/12.5/44.4/31.9/0.0
15	Initial	0.0/5.6/13.9/80.6/0.0	2.8/12.5/19.4/65.3/0.0	4.2/9.7/20.8/65.3/0.0
15	Update 600	0.0/5.6/9.7/84.7/0.0	0.0/5.6/18.1/76.4/0.0	2.8/8.3/18.1/70.8/0.0
21	Initial	0.0/0.0/0.0/100.0/0.0	0.0/0.0/0.0/100.0/0.0	0.0/0.0/0.0/100.0/0.0
21	Update 600	0.0/0.0/0.0/100.0/0.0	0.0/0.0/1.4/98.6/0.0	0.0/0.0/0.0/100.0/0.0

Appendix

Table 12: Final 3/9/18 automated comparison: percentages win/loss/order-disagreement/tie/missing for the final checkpoint versus each control. Each cell has 72 planned pairs, 24 families and both image orders. Missing judgments are retained in the denominator.

Branch	Control	IF	VC	VQ
9	Initial	11.1/4.2/22.2/62.5/0.0	12.5/6.9/38.9/41.7/0.0	11.1/27.8/48.6/12.5/0.0
9	Update 600	6.9/2.8/23.6/66.7/0.0	6.9/9.7/27.8/55.6/0.0	16.7/11.1/40.3/31.9/0.0
15	Initial	6.9/4.2/16.7/72.2/0.0	5.6/11.1/27.8/55.6/0.0	5.6/16.7/19.4/58.3/0.0
15	Update 600	4.2/2.8/6.9/86.1/0.0	2.8/5.6/20.8/70.8/0.0	2.8/15.3/13.9/68.1/0.0
21	Initial	0.0/0.0/5.6/94.4/0.0	1.4/2.8/18.1/77.8/0.0	0.0/0.0/11.1/88.9/0.0
21	Update 600	0.0/0.0/1.4/98.6/0.0	1.4/0.0/9.7/88.9/0.0	0.0/1.4/2.8/95.8/0.0

Appendix

Table 13: As Table 12 , schedule 6/12/12.

Branch	Control	IF	VC	VQ
9	Initial	5.6/22.2/26.4/45.8/0.0	11.1/26.4/36.1/26.4/0.0	16.7/36.1/40.3/6.9/0.0
9	Update 600	6.9/9.7/23.6/59.7/0.0	11.1/22.2/25.0/41.7/0.0	8.3/23.6/51.4/16.7/0.0
9	Native suffix	25.0/33.3/22.2/19.4/0.0	33.3/29.2/31.9/5.6/0.0	22.2/37.5/40.3/0.0/0.0
15	Initial	2.8/12.5/15.3/69.4/0.0	4.2/22.2/31.9/41.7/0.0	4.2/19.4/41.7/34.7/0.0
15	Update 600	4.2/6.9/18.1/70.8/0.0	4.2/18.1/30.6/47.2/0.0	5.6/16.7/34.7/43.1/0.0
15	Native suffix	6.9/23.6/29.2/40.3/0.0	18.1/22.2/43.1/16.7/0.0	18.1/37.5/31.9/12.5/0.0

Appendix

Table 14: As Table 12 , schedule 9/12/9.

Schedule	Valid masks	$\Delta$ BG LPIPS [95% CI]	$\Delta$ semantic proxy [95% CI]
3/9/18	63/72	-0.1248 [-0.1588, -0.0936]	-0.0046 [-0.0086, -0.0010]
6/12/12	61/72	-0.0924 [-0.1115, -0.0740]	-0.0037 [-0.0070, -0.0005]
9/12/9	60/72	-0.0696 [-0.0948, -0.0458]	-0.0007 [-0.0039, 0.0029]

Appendix

Table 15: Paired branch-9-minus-regeneration differences with family-bootstrap 95% intervals. Negative LPIPS differences indicate better preservation; negative semantic differences indicate a smaller CLIP margin change.

Figure 7: All pre-specified branch points under the final edit-only checkpoints. Each curve shows a preservation–semantic-proxy tradeoff, not a validated success frontier. Background means are conditional on valid source masks; semantic means use all attempts.

Branch	Native	Trained	Lower in	$\Delta$ [descriptive 95% interval]
9	0.2528	0.2296	41/60	$-0.0216$ [ $-0.0461$ , $-0.0036$ ]
15	0.1134	0.1134	28/60	$+0.0000$ [ $-0.0069$ , $+0.0069$ ]
21	0.0305	0.0398	8/60	$+0.0090$ [ $+0.0062$ , $+0.0119$ ]

Appendix

Table 16: Matched-prefix preservation at the final checkpoint. Background LPIPS uses historical source masks: 60/72 cases from 21 families per branch. Lower is better. The difference and interval give an equally weighted family estimate, whereas the two means average valid attempts.

Panel of Figure 8 — Figure 8: Selected differences beyond prefix reuse. Both suffixes start from the same state. The trained output better retains the person’s scale and the wall in the first row, and the striped clothing and portrait scale in the second. The bicycle caption also requests napping, which neither edit realizes. Post-hoc examples from the final 9/12/9 compiler, uncropped and unretouched.

Schedule	pad	$n$	Regen.	FlowEdit	SDEdit	Native
3-9-18	0%	68/72	$-0.122$	$+0.100$	$-0.040$	$-0.016$
3-9-18	5%	65/72	$-0.129$	$+0.097$	$-0.052$	$-0.017$
3-9-18	10%	62/72	$-0.127$	$+0.097$	$-0.056$	$-0.016$
6-12-12	0%	66/72	$-0.092$	$+0.105$	$-0.043$	$-0.024$
6-12-12	5%	62/72	$-0.094$	$+0.104$	$-0.055$	$-0.022$
6-12-12	10%	60/72	$-0.090$	$+0.100$	$-0.064$	$-0.023$

Appendix

Table 17: Source-box background LPIPS on the full evaluation cohort. Entries are paired mean differences (ours branch 9 minus comparator; lower is better).

n

is defined/attempted, with missing cases retained.

Schedule	pad	$n$	Regen.	FlowEdit	SDEdit	Native
3-9-18	0%	33/36	$-0.114$	$+0.114$	$-0.023$	$-0.011$
3-9-18	5%	30/36	$-0.124$	$+0.111$	$-0.043$	$-0.014$
3-9-18	10%	30/36	$-0.127$	$+0.107$	$-0.048$	$-0.012$
6-12-12	0%	33/36	$-0.083$	$+0.109$	$-0.035$	$-0.029$
6-12-12	5%	30/36	$-0.083$	$+0.106$	$-0.058$	$-0.031$
6-12-12	10%	30/36	$-0.083$	$+0.097$	$-0.070$	$-0.032$

Appendix

Table 18: As Table 17 , on the eligible cohort.

Comparator	Schedule	pad	$n$	$\Delta$ [95% interval]
Regeneration	3-9-18	0%	68/72	$-0.122\,[-0.155,-0.092]$
Regeneration	3-9-18	5%	65/72	$-0.129\,[-0.168,-0.095]$
Regeneration	3-9-18	10%	62/72	$-0.127\,[-0.170,-0.090]$
Regeneration	6-12-12	0%	66/72	$-0.092\,[-0.113,-0.073]$
Regeneration	6-12-12	5%	62/72	$-0.094\,[-0.116,-0.072]$
Regeneration	6-12-12	10%	60/72	$-0.090\,[-0.110,-0.070]$

Appendix

Table 19: Selected 95% family-bootstrap intervals for the full evaluation cohort. All comparator intervals and RGB MSE are available in the analysis artifact.

Schedule	Branch	pad 0%	pad 5%	pad 10%
3-9-18	9	$-0.016$ (68/72)	$-0.017$ (65/72)	$-0.016$ (62/72)
3-9-18	15	$+0.003$ (68/72)	$+0.003$ (65/72)	$+0.003$ (62/72)
3-9-18	21	$+0.001$ (68/72)	$+0.001$ (65/72)	$+0.001$ (62/72)
6-12-12	9	$-0.024$ (66/72)	$-0.022$ (62/72)	$-0.023$ (60/72)
6-12-12	15	$-0.002$ (66/72)	$-0.001$ (62/72)	$-0.000$ (60/72)
6-12-12	21	$+0.003$ (66/72)	$+0.003$ (62/72)	$+0.003$ (60/72)

Appendix

Table 20: Native shared-path source-box LPIPS comparison, full evaluation cohort. Point differences are ours minus native;

n

is defined/attempted.

Schedule	Branch	pad 0%	pad 5%	pad 10%
3-9-18	9	$-0.011$ (33/36)	$-0.014$ (30/36)	$-0.012$ (30/36)
3-9-18	15	$+0.003$ (33/36)	$+0.003$ (30/36)	$+0.003$ (30/36)
3-9-18	21	$+0.002$ (33/36)	$+0.002$ (30/36)	$+0.002$ (30/36)
6-12-12	9	$-0.029$ (33/36)	$-0.031$ (30/36)	$-0.032$ (30/36)
6-12-12	15	$-0.002$ (33/36)	$-0.002$ (30/36)	$-0.002$ (30/36)
6-12-12	21	$+0.003$ (33/36)	$+0.003$ (30/36)	$+0.003$ (30/36)

Appendix

Table 21: As Table 20 , on the eligible cohort.

Schedule	Recomputed (s)	Cached (s)	Speedup	Time saved
3/9/18	9.28	7.19	1.29 $\times$	22.6%
6/12/12	9.20	7.18	1.28 $\times$	22.0%
9/12/9	9.39	7.19	1.31 $\times$	23.4%

Appendix

Table 22: Mean time over four scenes and three repetitions to produce four target continuations. Every cached endpoint matches its recomputed counterpart exactly.

Training schedule	3/9/18	6/12/12	9/12/9
3/9/18	0.074 (24) / 0.129 (24)	0.070 (24) / 0.133 (23)	0.075 (24) / 0.139 (22)
6/12/12	0.076 (24) / 0.138 (24)	0.073 (24) / 0.138 (23)	0.078 (24) / 0.144 (22)
9/12/9	0.077 (24) / 0.160 (24)	0.070 (24) / 0.144 (23)	0.074 (24) / 0.143 (22)
Initialization	0.072 (24) / 0.126 (24)	0.073 (24) / 0.134 (23)	0.076 (24) / 0.143 (22)

Appendix

Table 23: Complete crossing, shown at branch 15. Cells report CLIP paired margin change / background LPIPS, with defined denominators in parentheses (24 attempted). These proxies do not establish edit success.

Schedule; branch	Matched source	Own source
3/9/18; 0	0.101 (24) / 0.399 (24)	0.101 (24) / 0.390 (22)
3/9/18; 9	0.091 (24) / 0.266 (24)	0.092 (24) / 0.254 (22)
3/9/18; 15	0.074 (24) / 0.129 (24)	0.074 (24) / 0.126 (22)
3/9/18; 21	0.019 (24) / 0.036 (24)	0.022 (24) / 0.038 (22)
6/12/12; 0	0.095 (24) / 0.375 (23)	0.095 (24) / 0.369 (22)
6/12/12; 9	0.089 (24) / 0.272 (23)	0.088 (24) / 0.260 (22)

Appendix

Table 24: Diagonal source-path comparison. CLIP paired margin change / background LPIPS (defined denominators); different source modes use different source images.

Compiler	B0	B1	B2	all	nochange
Trained 3/9/18	0.001 (24) 0.179 (24)	0.015 (24) 0.306 (24)	0.090 (24) 0.237 (24)	0.101 (24) 0.399 (24)	-0.000 (24) 0.147 (24)
Trained 6/12/12	0.001 (24) 0.204 (23)	0.040 (24) 0.301 (23)	0.048 (24) 0.114 (23)	0.095 (24) 0.375 (23)	0.000 (24) 0.070 (23)
Trained 9/12/9	0.004 (24) 0.221 (22)	0.059 (24) 0.245 (22)	0.027 (24) 0.094 (22)	0.093 (24) 0.336 (22)	0.000 (24) 0.078 (22)
Init. 3/9/18	0.000 (24) 0.088 (24)	0.012 (24) 0.319 (24)	0.087 (24) 0.191 (24)	0.099 (24) 0.411 (24)	0.000 (24) 0.000 (24)
Init. 6/12/12	-0.000 (24) 0.190 (23)	0.039 (24) 0.312 (23)	0.047 (24) 0.078 (23)	0.093 (24) 0.382 (23)	0.000 (24) 0.000 (23)
Init. 9/12/9	0.006 (24) 0.208 (22)	0.060 (24) 0.256 (22)	0.026 (24) 0.039 (22)	0.094 (24) 0.343 (22)	0.000 (24) 0.000 (22)

Appendix

Table 25: Block interventions at branch 0, where all blocks are active. CLIP paired margin change / background LPIPS (defined denominators). Initial and trained methods share the frozen source; these are descriptive measurements.

Figure 9: Generation along the learned path. Final compiler, identical initial noise, with source, subject-changed and background-changed captions. Columns show $\widehat{x}_{0}=x_{t}-\sigma_{t}v_{\theta}(x_{t},c_{k})$ from the last executed denoiser evaluation in each block (steps 6, 18, 30), not decodes of noisy states. B0 exposes coarse scene appearance, B1 makes the subject recognizable and B2 completes the image. Changing the scene also changes pose and appearance, so stage progression alone is not factor independence.

Figure 10: Edit strength against source closeness, all five starts. Each curve traces one method across starts 0, 6, 9, 15 and 21, for the whole cohort and the two non-empty source-size strata. Later starts stay closer to the source but edit less, so low LPIPS at a late start indicates a weak edit. Subject panels use both-detector strict replacement, and background panels the uncalibrated CLIP scene margin. The medium stratum holds only eight sources.

Method	DINO	OWLv2	Both	BG LPIPS ( $n$ )
B0-isolated learned	19	48	19	0.173 (48)
Native shared state	9	48	9	0.451 (48)
FlowEdit PNG, CFG4	9	48	9	0.297 (48)
FlowEdit PNG, CFG6	11	48	11	0.355 (48)
FlowEdit PNG, CFG8	13	48	13	0.397 (48)
FlowEdit latent, CFG4	8	48	8	0.293 (48)

Appendix

Table 26: B0-isolated source cohort: subject replacement. Strict category replacement requires source category present in the source, target present after editing and source category absent after editing. Counts use all 48 attempts; Both requires both calibrated detectors. BG LPIPS uses 5% source-box padding; parentheses give defined backgrounds. Lower LPIPS is useful only with successful editing.

Method	DINO	OWLv2	CLIP	FG LPIPS ( $n$ )
B0-isolated learned	48	48	0.103	0.657 (48)
Native shared state	48	48	0.101	0.664 (48)
FlowEdit PNG, CFG4	48	48	0.089	0.572 (48)
FlowEdit PNG, CFG6	48	48	0.092	0.605 (48)
FlowEdit PNG, CFG8	48	48	0.094	0.642 (48)
FlowEdit latent, CFG4	48	48	0.088	0.577 (48)

Appendix

Table 27: B0-isolated source cohort: background changes. Retained counts require the original subject category in both source and output, using all 48 attempts. CLIP denotes scene-margin change; FG LPIPS uses 5% padded source boxes, with defined-region counts in parentheses.

Direction	Start	Both/retained (pp)	95% interval	Region LPIPS
subject	0	+20.8	[+4.2, +45.8]	-0.277
subject	6	+6.2	[+0.0, +18.8]	-0.007
subject	9	+8.3	[+2.1, +14.6]	-0.001
subject	15	+8.3	[+2.1, +16.7]	-0.001
subject	21	+4.2	[+0.0, +10.4]	-0.001
background	0	+0.0	[+0.0, +0.0]	-0.007

Appendix

Table 28: B0-isolated learned minus native shared state, all 48 paired cases. Detector entries are percentage-point differences in strict replacement (subject) or source-subject retention (background); LPIPS entries are differences in the 5% padded region metric, where a negative value is closer to the source. Intervals are unadjusted descriptive family bootstrap percentiles.

Branch	Method	D/O/B	BG $\downarrow$	FG $\downarrow$	Scene $\Delta$
9	Learned suffix	13/48/13	0.1195	0.2953	0.0844
9	Native shared-state	8/48/8	0.1210	0.3095	0.0826
9	FlowEdit PNG, CFG 4	10/48/10	0.0762	0.1890	0.0597
9	FlowEdit PNG, CFG 6	14/48/14	0.1059	0.2424	0.0816
9	FlowEdit PNG, CFG 8	12/48/12	0.1407	0.3065	0.0869
9	FlowEdit latent, CFG 4	10/48/10	0.0697	0.2002	0.0637

Appendix

Table 29: Complete bidirectional grid, 48 attempts per direction and row. D/O/B are strict replacement counts for DINO, OWLv2 and both. BG and FG are non-target LPIPS for subject and background edits (lower is better), and Scene

\Delta

is the CLIP margin change.

Figure 11: Bidirectional stress-test tradeoffs for all sources and for large source boxes. Lines connect the three measured branch points. Upper panels use strict replacement, lower panels the scene-direction CLIP change.

Panel of Figure 12 — Figure 12: Prefix reuse retains what regeneration discards. Regeneration adds a second rider, reframes the plants and replaces the room and the lettering. Prefix reuse keeps the single rider and herd, the potted plants and their layout, and the candlelit room, the bow tie and the “me to you” lettering, while changing the requested subject.

Diffusion-Based Image Generation

Diffusion-based image generation creates images by progressively denoising noise, with controls for content, style, and structure.

See all 328 papers

DuET: Dual Expert Trajectories for Diffusion Image Editing
Jun 11, 2026Lidia Troeshestova, Alexander Ustyuzhanin, Sergey KastryulinText-Guided Image Editing Diffusion-Based Image Editing
HSE University · Yandex
PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents
Sep 28, 2026Vighnesh Subramaniam, Boris Katz, Brian Cheung +3Diffusion Model Guidance T2I Generation
MIT CSAIL · Google Cloud AI Research · UCSF
RegionCache: Semantic-Aware Region Reuse for Efficient Multi-Turn Image Generation
Aug 30, 2026Peizheng Li, Xin Ai, Hanyuan Liu +2Multi-Turn Image Editing Diffusion Model Caching
School of Computer Science and Engineering, Northeastern University, China · School of Computer Science

Jun 11, 2026J/K move · Enter open · S save

DuET: Dual Expert Trajectories for Diffusion Image Editing

Lidia Troeshestova, Alexander Ustyuzhanin, Sergey Kastryulin

HSE University · Yandex

Text-Guided Image Editing Diffusion-Based Image Editing Conditional Diffusion Models Training-Free Image Editing Diffusion-Based Denoising

Open paper AskPDF arXiv

Recent diffusion editors perform diverse instruction-based edits while conditioning on the source image at every denoising step. Yet persistent source-image conditioning can limit how fully an edit is executed and how natural the result appears, especially when the target scene diverges substantially from the input. We introduce DuET (Dual Expert Trajectories), a training-free inference method that temporarily relaxes source-image conditioning by transitioning through a text-to-image phase before returning to edit mode, allowing the denoising trajectory to move toward the target distribution while retaining the structural benefits of image-conditioned editing. Without modifying model weights or increasing sampling cost, DuET consistently improves instruction relevance, semantic fidelity, and perceptual quality across diverse models and benchmarks. In some cases, these gains come with a modest reduction in source-image preservation, revealing a predictable trade-off between source preservation and edit fidelity.

Abstract

Diffusion-Based Image Generation

Explore similar work

DuET: Dual Expert Trajectories for Diffusion Image Editing

PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents

RegionCache: Semantic-Aware Region Reuse for Efficient Multi-Turn Image Generation