cs.CVOct 1, 2026

When Text-to-Image Helps Editing: The Effects of Conditioning During Denoising

Authors: Lidia Troeshestova, Alexander Ustyuzhanin, Sergey Kastryulin

Organizations: HSE University & Yandex · Yandex

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Unified models are trained for both instruction-based image editing and text-to-image (T2I) generation, but standard editing pipelines keep source-image conditioning throughout denoising. We ask whether editing can benefit from T2I, and study how the effects of conditioning vary across edits and denoising stages. In pure editing, source attention declines for some edits over the sampling trajectory. This observation led us to task switching, which lets the model draw on its T2I capabilities. Across three unified editors and four benchmarks, switching to the T2I task for bounded intervals improves edit quality, while mean perceptual preservation remains close to pure editing across all three models. Unified editors therefore benefit from using both conditioning modes they are trained for, and the timing of the switch sets the balance between quality and preservation.

Figures & tables

Figure 1: Switching conditioning helps added objects fit the scene. Adding two dogs to a blue-lit room with pure editing produces artifacts and mismatched lighting (orange close-up). Switching to caption-only T2I at k1k_{1} improves integration but changes the room. The E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E} schedule returns to editing at k2k_{2} to retain more of the source.
Figure 2: Source attention varies across edits (FLUX2-Klein 4B, step 9). Maps average over heads and blocks. aia_{i} @9 shows per-patch source attention; CV@9 shows the coefficient of variation in local 3×33\times 3 neighborhoods.
Figure 3: Conditioning disagreement and source attention along pure editing (FLUX2-Klein 4B). (a) Source-attention mass generally decreases, while local edits retain high attention late in sampling. (b) Prediction cosine similarity, initially low (median 0.480.48 ). (c) T2I/edit prediction norm ratio. Both are evaluated at identical latent states.
Mode / controlImageText
E\mathrm{E}xsrcx_{\mathrm{src}}peditp_{\mathrm{edit}}
E∗\mathrm{E}^{*}xsrcx_{\mathrm{src}}pedit∗p_{\mathrm{edit}}^{*}
T2I\mathrm{T2I}∅\varnothingctgtc_{\mathrm{tgt}}
T2I\mathrm{T2I} w/ editing prompt∅\varnothingpeditp_{\mathrm{edit}}
E\mathrm{E} w/ captionxsrcx_{\mathrm{src}}ctgtc_{\mathrm{tgt}}
Table 1: Conditioning inputs.
  GEditImgEditMagicBrushGIE-Bench
       Func. Corr.Preservation
ModelSamplingSCPQOoverallCLIP-TOverallSSIMGPT-4.1
FLUX2-Klein 4B±\pm std (3 seeds)0.050.030.040.040.0010.350.0010.05
 Pure editing7.727.467.113.740.31773.400.8057.65
 Fixed8.087.907.673.990.32276.460.7817.57
 Selective ∗8.017.677.453.880.31977.110.7867.60
Table 2: A fixed T2I interval improves quality across models and benchmarks. Fixed and selective switching use the six-step interval [10,16)[10,16) . The first six scores measure edit quality. Higher is better throughout. Green/red intensity shows the magnitude of increases/decreases from each model’s pure-editing baseline. Standard deviations describe baseline variability across three seeds. GPT-4.1 preservation is scored on a 00 – 1010 scale.
Figure 4: A fixed T2I interval can improve both GEdit semantic consistency and perceptual quality on each model. GEdit scores use the GPT-4.1 judge. Points are E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E} schedules, labeled by [k1,k2)[k_{1},k_{2}) . The shaded region indicates improvement over pure editing on both metrics.
Figure 5: Returning to edit conditioning retains more of the source (FLUX2-Klein 4B). A one-way T2I switch can improve instruction fidelity while changing unrelated content; resuming edit conditioning after the interval limits these changes.
Figure 6: Removing the source image allows larger quality gains with the same caption (GEdit, Qwen3.5). Bands span per-metric extrema across switch timings, with and without a return to editing, so one band edge can combine several configurations. Axes normalized independently.
Figure 7: Conditioning and timing affect edit quality (FLUX2-Klein 4B, GEdit, Qwen3.5). Left: best- G_O\mathrm{G\_O} configurations per conditioning group over shared placements in Tables 7 and 7 ; red: pure editing. Colors denote text schedules; solid/dashed lines indicate source kept/removed in the interval. Axes use independent linear scales above and below pure editing. Right: T2I sweeps by duration Δk\Delta k ; shading marks gains in both metrics.
Figure 8: Preservation and correctness on GIE-Bench (FLUX2-Klein 4B). Left: mean GPT-4.1 preservation versus SSIM outside the edit mask; shading: near-baseline plateau; faint curve: visual guide. Pearson rr is computed separately below and above SSIM 0.760.76 . Right: Gemini-3-Flash correctness versus mean GPT preservation with 95%95\% source-clustered confidence intervals ( Appendix F ).
Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Missed edits (FLUX2-Klein 4B, GEdit). Pure editing leaves the requested change largely unexecuted. A one-way switch uses T2I from step 10 onward; E →\to T2I →\to E returns to editing after [10,16)[10,16) .
Figure 10: Incorrect edits (FLUX2-Klein 4B, GEdit). Pure editing changes the image but does not produce the requested result. Columns as in Figure 9 .
Figure 11: Unnatural and incomplete edits (FLUX2-Klein 4B, GEdit). The first four rows illustrate unnatural appearance or artifacts under pure editing; the final row illustrates an incomplete transformation. Columns as in Figure 9 .
  Functional correctnessPreservation
ScheduleIntervalAdd ObjectAttribute ChangeColor ChangeLayout Modif.Object ReplaceRemove ObjectScene / Bkgd ChangeSize ChangeTextual EditOverallSSIM ↑\uparrowCLIP ↑\uparrowUnmasked CLIP ↑\uparrowPSNR ↑\uparrowMSE ↓\downarrow
Baseline—91.6788.3390.7644.1793.3368.3395.8320.8367.5073.400.8050.970.91320.881882.32
E→E∗→E\mathrm{E}\to\mathrm{E}^{*}\to\mathrm{E}[0,10)[0,10)91.6793.3395.8043.3393.3371.4396.6727.561.6774.950.8010.9700.91120.322117.10
E→E∗→E\mathrm{E}\to\mathrm{E}^{*}\to\mathrm{E}[25,35)[25,35)89.5288.0092.8645.4594.2966.3493.4823.5867.6573.130.810.970.91420.761897.70
E→E∗→E\mathrm{E}\to\mathrm{E}^{*}\to\mathrm{E}[35,45)[35,45)90.8389.1791.6048.3393.2867.5095.8325.0064.1773.930.810.970.91320.841878.95
E ∗[0,50)[0,50)90.8393.3394.1241.679570.5996.6730.837075.880.7990.9700.91020.252174.08
Appendix
Table 3: GIE-Bench results (FLUX2-Klein 4B, GPT-4o correctness judge). Functional correctness by edit category and source-preservation metrics for fixed schedules, the improved-instruction control, and selective application.
Table 15
Table 16
ScheduleImageTextBest SCBest PQBest O
E\mathrm{E}(x,x,x)(x,x,x)(p,p,p)(p,p,p)7.917.657.33
E→E∗→E\mathrm{E}\to\mathrm{E}^{*}\to\mathrm{E}(x,x,x)(x,x,x)(p,p∗,p)(p,p^{*},p)8.50 ([0,10))([0,10))7.69 ([0,10))([0,10))7.80 ([0,10))([0,10))
Caption in interval; image retained(x,x,x)(x,x,x)(p,c,p)(p,c,p)8.14 ([15,50))([15,50))8.18 ([0,50))([0,50))7.58 ([10,50))([10,50))
Image off in interval; original instruction(x,∅,x)(x,\varnothing,x)(p,p,p)(p,p,p)7.95 ([45,50))([45,50))8.38 ([5,50))([5,50))7.39 ([45,50))([45,50))
Caption throughout; image off in interval(x,∅,x)(x,\varnothing,x)(c,c,c)(c,c,c)7.58 ([40,50))([40,50))8.36 ([0,50))([0,50))7.53 ([10,20))([10,20))
E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E}(x,∅,x)(x,\varnothing,x)(p,c,p)(p,c,p)8.07 ([25,35))([25,35))8.36 ([0,45))([0,45))7.69 ([20,30))([20,30))
Appendix
Table 8: GEdit best scores over the displayed placements (FLUX2-Klein 4B, Qwen3.5). Each row pools interval and single-switch results for one conditioning schedule. Metrics are optimized separately; parentheses give a maximizing interval, so entries in a row need not describe one configuration. Image/text triples specify conditioning before, during, and after the interval. Forward and reverse switches are written as [k,50)[k,50) and [0,k)[0,k) , respectively. Ties use the earliest start, then the earliest end.
ScheduleIntervalbackgroundadjuststyleextractremoveaddreplacecomposeactionoverall
FLUX2-Klein 4B
Baseline—4.213.854.801.472.994.404.183.264.703.74
E→E∗→E\mathrm{E}\to\mathrm{E}^{*}\to\mathrm{E}[0,10)[0,10)4.113.634.803.744.334.494.163.684.734.18
E→E∗→E\mathrm{E}\to\mathrm{E}^{*}\to\mathrm{E}[25,35)[25,35)4.083.834.741.382.984.364.143.194.913.69
E→E∗→E\mathrm{E}\to\mathrm{E}^{*}\to\mathrm{E}[35,45)[35,45)4.153.814.801.553.014.404.203.264.803.73
E ∗[0,50)[0,50)4.083.734.753.644.284.424.193.554.734.15
Appendix
Table 9: ImgEdit per-category scores (GPT-4o judge). Category breakdown for FLUX2-Klein 4B, 9B, and BAGEL: baseline, improved-instruction controls, T2I schedules, and selective application where reported.
 GEditImgEditMagicBrushGIE-Bench
SamplingSCPQOoverallCLIP-TFCSSIMU-CLIP
FLUX2-Klein 4B
±\pm std (3 seeds)0.050.030.040.040.0010.350.0010.002
Pure E7.727.467.113.740.31773.400.8050.913
E ∗ ⁣→{}^{*}\!\to E (10)(10)8.167.497.544.180.31774.950.8010.911
E ∗8.177.497.544.150.31775.880.7990.910
Appendix
Table 10: Improved-instruction controls and best observed T2I intervals across three models. The evaluation protocols match Table 2 : GPT-4.1 for GEdit and GPT-4o for ImgEdit and GIE correctness. The best E →\to T2I →\to E score is selected separately for each column from the available interval sweep; the row below reports one winning interval. These maxima need not share one configuration, and sweep coverage varies by model and benchmark. Higher is better; green/red intensity shows changes from each model’s pure-editing baseline, using the per-metric scales of Table 2 . The ±\pm rows are three-seed baseline standard deviations, as in Table 2 .
Figure 12: Continuous guidance in the factorial weight plane (FLUX2-Klein 4B). We fix wt=4w_{t}=4 and apply each weight pair throughout 50 denoising steps. Multimodal guidance follows the diagonal (wi,wti)=(α,4α)(w_{i},w_{ti})=(\alpha,4\alpha) , interpolating from caption-only T2I at α=0\alpha=0 to editing at α=1\alpha=1 ; solid outlines mark 0<α<10<\alpha<1 . Image guidance follows the rightmost column, (wi,wti)=(α,4)(w_{i},w_{ti})=(\alpha,4) ; dashed outlines mark 0≤α<10\leq\alpha<1 . Even at α=0\alpha=0 , image guidance uses the image-conditioned predictions EE and II . Numbers indicate forward passes per denoising step: four for intermediate multimodal guidance, three for image guidance with α<1\alpha<1 , and two for the T2I and editing endpoints.
ConditioningForwards / stepRequired predictions
Pure E (baseline)2E, IE,\,I
Pure T2I2T, UT,\,U
Image guidance ( α<1\alpha<1 )3E, I, UE,\,I,\,U
Multimodal guidance ( 0<α<10<\alpha<1 )4E, T, I, UE,\,T,\,I,\,U
Appendix
Table 11: Forward passes within the denoising trajectory (FLUX2-Klein 4B/9B). Counts are per sampling step, counting batched conditioning branches separately. Pure editing and the pure T2I endpoint each require two passes.
  GEditImgEditMagicBrushGIE-Bench
α\alphaFwd/ stepSCPQOoverallCLIP-TFCSSIMU-CLIP
Image guidance
Interval [10,20)[10,20)
0.00.037.697.277.103.660.31974.840.7820.891
0.250.2537.747.367.153.710.31974.980.7900.898
0.50.537.707.437.153.730.31973.350.7970.904
Appendix
Table 12: Image guidance and multimodal guidance (FLUX2-Klein 4B). GEdit uses GPT-4.1; ImgEdit and GIE correctness use GPT-4o. α=1\alpha=1 is editing. For multimodal guidance, α=0\alpha=0 is the T2I endpoint; for image guidance, it sets the image guidance scale to zero. Shading uses one baseline-centered scale per metric across all blocks.
SamplingCaption modelImage generationVLM callTotal
Pure editing—5.325—5.475
E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E}Gemini-2.5-pro4.90026.76031.810
E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E}Qwen3.5-9B4.9000.4605.510
E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E}Qwen3.5-27B4.9001.1436.193
E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E}Qwen3.5-35B-A3B4.9000.2395.289
Appendix
Table 13: End-to-end time per edit, including caption generation. The E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E} schedule uses the interval [10,20)[10,20) . Timings use FLUX2-Klein 4B on 1,0001{,}000 samples; all times are means in seconds. For Gemini, the VLM-call time is the API-call latency. In addition to image generation and the VLM call, total time includes VAE encoding and decoding and image resizing.
 GEditImgEditMagicBrushGIE-Bench
Caption modelOOverallCLIP-TFC (%)SSIM
±\pm std (3 seeds)0.040.040.0010.350.001
Pure editing (no caption)7.113.740.31773.400.805
Gemini-2.5-pro7.534.160.32476.740.684
Qwen3.5-27B7.514.100.32477.020.684
Qwen3.5-35B-A3B7.484.110.32377.370.683
Appendix
Table 14: Captioner choice for E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E} on [10,20)[10,20) (FLUX2-Klein 4B). The interval matches the timing measurements in Table 13 . Pure editing reproduces Table 2 ; Gemini-caption results use the same [10,20)[10,20) interval as the Qwen results. Caption models supply target captions; the benchmark metrics follow Section 4 . Higher is better. Green/red intensity shows gains/losses from pure editing, normalized by the largest absolute change per metric within this table. The ±\pm row gives the three-seed standard deviation of the pure-editing baseline.
ModelRoutingPreservationFidelityPQO
FLUX2-Klein 4BPure editing8.9028.3957.4217.112
Attention rule ( 68%68\% switched)8.8608.6967.6987.459
Random, preservation-matched ( p=0.33p=0.33 )8.8608.5467.5787.295
Difference00+0.149+0.149+0.120+0.120+0.164+0.164
FLUX2-Klein 9BPure editing9.1279.0337.7467.778
Attention rule ( 69%69\% switched)9.0859.2017.8697.949
Appendix
Table 15: Attention rule versus preservation-matched random routing (GEdit, GPT-4.1, macro averaging). Random routing sends each edit to the [10,16)[10,16) interval with a probability chosen to match the rule’s mean preservation. Differences are rule minus random; positive favors the rule.
 GPT preservationSSIM
ComparisonΔ\Delta95% CIppΔ\Delta95% CIpp
FLUX2-Klein 4B: comparisons against pure editing
E→T2I\mathrm{E}\to\mathrm{T2I} , k=35k=35−0.059-0.059[−0.207,+0.088][-0.207,+0.088]0.9460.946−0.0371-0.0371[−0.0404,−0.0338][-0.0404,-0.0338]<0.001<0.001
E→T2I\mathrm{E}\to\mathrm{T2I} , k=40k=40+0.099+0.099[−0.046,+0.244][-0.046,+0.244]0.3910.391−0.0187-0.0187[−0.0207,−0.0168][-0.0207,-0.0168]<0.001<0.001
E→T2I\mathrm{E}\to\mathrm{T2I} , k=45k=45+0.093+0.093[−0.043,+0.228][-0.043,+0.228]0.3910.391−0.0053-0.0053[−0.0061,−0.0044][-0.0061,-0.0044]<0.001<0.001
E→T2I→E\mathrm{E}\to\mathrm{T2I}\to\mathrm{E} , [25,35)[25,35)−0.341-0.341[−0.521,−0.161][-0.521,-0.161]<0.001<0.001−0.0567-0.0567[−0.0613,−0.0521][-0.0613,-0.0521]<0.001<0.001
Appendix
Table 16: Paired preservation effects on GIE-Bench. Positive differences indicate better preservation. Shading marks p<0.05p<0.05 (green: gain; red: loss). Fixed and selective application use [10,16)[10,16) . For each metric, Bonferroni 95%95\% confidence intervals and Holm-adjusted pp values account for multiple comparisons within the 4B baseline sweep and, separately, the pooled 9B/BAGEL comparisons. † The exploratory 4B selective–fixed comparison uses an ordinary 95%95\% interval and an unadjusted pp value.

Explore similar work

CardsList

  1. DuET: Dual Expert Trajectories for Diffusion Image Editing

    Jun 11, 2026Lidia Troeshestova, Alexander Ustyuzhanin, Sergey KastryulinDiffusion-Based Image EditingImage Editing

    HSE University · Yandex

  2. Diffusion Image Editing via Asynchronous Token Decoding

    Aug 10, 2026Yang Shi, Liangsi Lu, Minzhe Guo +4Diffusion-Based Image EditingText-To-Image Diffusion Models

    Guangdong University of Technology Guangzhou, China · Hong Kong Baptist University Hong Kong, China · Peking University Beijing, China +1

  3. DirectEdit: Step-Level Accurate Inversion for Flow-Based Image Editing

    May 4, 2026Desong Yang, Mang YeDiffusion-Based Image EditingImage Editing

    National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, China.