Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion

Jinjian Jiang; Jing Peng; Rehan Deen; Yu Cao; Ziquan Liu; Muhammad Salman Haleem; David Mguni

cs.LGOct 22, 2026

Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion

Authors: Jinjian Jiang, Jing Peng, Rehan Deen, Yu Cao, Ziquan Liu, Muhammad Salman Haleem, David Mguni

Organizations: University of Warwick · Columbia University · Independent Researcher · Queen Mary University of London

Abstract

Many deployments require a model to move rapidly from a data-rich source to a related target observed only sparsely. This is especially consequential where target data are difficult to collect, as in healthcare settings with persistent population or acquisition imbalances. Retraining or fine-tuning a high-capacity generator for every target is costly and can be statistically unreliable when target data are scarce. We introduce Meta-Diff, which instead learns a recurring source--target relation once in a compact task space. A shared encoder maps each distribution to a low-dimensional coordinate; a learned relation transports a new source coordinate to its predicted target, and a shared diffusion backbone generates from that prediction without target-specific retraining. Controlled Gaussian-mixture experiments identify when this works: task identity must survive the representation, the relation must generalise across tasks, and the source--target shift must leave useful headroom for transport. On held-out CIFAR-100 validation tasks, correct source $+$ transport achieves 13.9% lower mean SW than transport from the wrong source. With validation-selected adaptation settings in the final-test one-target comparison, its mean SW is 14.7% lower than target-only modelling and 32.7% lower than full fine-tuning; the main route itself uses no target image and assumes the applicable relation is known. On matched CIFAR test episodes, generating from the clean-source coordinate and then applying the known corruption yields lower SW than task-space transport. Office-Home exposes a complementary boundary: transport can align target-domain statistics without reliable object identity. Together, these results recast extreme-few-shot adaptation as reusable transport in task space, while making the conditions for its success and failure directly measurable.

Figures & tables

Panel of Figure 1 — Figure 1: Meta-Diff learns a relation between task distributions, rather than a separate adaptation for each target. Meta-training observes the same source $\rightarrow$ target relation across different semantic tasks; a shared encoder embeds these distributions in task space; and the learned relation is applied to the source coordinate of a held-out task. At every headline $K_{T}$ , the main route generates directly from the transported coordinate, with no target images, coordinate refinement or backbone update. Target-support budgets apply to independent baselines.

Figure 2: Qualitative task-coordinate control on held-out CIFAR generations. Columns share initial diffusion noise; rows differ only in the task coordinate (own, mean, shuffled, or $z=0$ ). The samples come from the 50k-step CIFAR checkpoint, with matched tasks and paired noise. Own-task coordinates are encoder coordinates, not transported predictions; deployment claims use the separate paired route evaluations in Section 5.2 .

Panel of Figure 3 — Figure 3: GMM operating regime. Left: correct-source advantage $G_{\mathrm{source}}=\mathrm{SW}_{\mathrm{wrong}}-\mathrm{SW}_{\mathrm{correct}}$ over the full source-identifiability/relation-heterogeneity grid at $K_{T}=0$ . The advantage disappears when source identity is absent and attenuates as the relation becomes less predictable. Right: intrinsic Bayes task identification and nearest-centroid identification in the learned encoder coordinate, averaged over relation levels. Nearly perfect intrinsic identifiability can coexist with substantially weaker usable identity. Full cell-level results and paired intervals are in Appendix B .

Panel of Figure 4 — Figure 4: Two additional GMM boundaries. Left: task identity through the pipeline. Bayes-ID is the intrinsic ceiling, learned $z_{S}$ measures usable source identity, and transported $\widetilde{z}_{T}$ measures whether the predicted target coordinate retrieves the correct held-out target task; raw $z_{S}\!\to z_{T}$ retrieval is a cross-domain baseline. Right: the shift-magnitude sweep separates source specificity from the value of transport, with hierarchical seed/task 95% intervals. The correct source remains informative as $\lambda\!\to\!0$ , whereas the advantage of transport over source reuse grows only when there is nontrivial source–target headroom. Exact tables, regression diagnostics and uncertainty are in Appendix B .

A. Zero-target controls: 60 validation episodes
Route	SW	$\Delta$ SW	Class consistency	$\Delta$ class
correct source $+$ transport	$0.2061$ $[0.1789,0.2392]$	–	$0.1075$ $[0.0740,0.1479]$	–
Strict source reuse	$0.2416$ $[0.2169,0.2703]$	$0.0355^{\ast}$ $[0.0283,0.0420]$	$0.1102$ $[0.0726,0.1554]$	$-0.0028$ $[-0.0160,0.0083]$
Wrong source	$0.2393$ $[0.2085,0.2737]$	$0.0333^{\ast}$ $[0.0163,0.0525]$	$0.0470$ $[0.0366,0.0582]$	$0.0604^{\ast}$ $[0.0297,0.0977]$
Relation-only	$0.2407$ $[0.2115,0.2742]$	$0.0346^{\ast}$ $[0.0224,0.0477]$	$0.0490$ $[0.0382,0.0608]$	$0.0585^{\ast}$ $[0.0293,0.0932]$
B. Final-test SW: 180 episodes, 60 per budget

Table 1: CIFAR headline comparisons for correct source

+

transport,

J=0

. Panel A uses validation classes; Panel B uses separate final-test classes. Distances:

\Delta=\mathrm{comparator}-\mathrm{main}

; consistency:

\Delta=\mathrm{main}-\mathrm{comparator}

. Positive favours the main route. Entries show mean [95% class-cluster CI]; stars use unrounded paired intervals excluding zero, in either direction.

Paired SW comparison	$K_{T}=1$	$K_{T}=5$	$K_{T}=20$
FiLM $-$ basis	$-0.0094$ $[-0.0238,0.0046]$	$-0.0148$ $[-0.0298,0.0003]$	$-0.0149$ $[-0.0300,0.0003]$
Linear $-$ MLP (basis)	$0.0022$ $[-0.0031,0.0080]$	$0.0004$ $[-0.0056,0.0068]$	$0.0036$ $[-0.0019,0.0097]$
MLP $-$ linear (FiLM)	$0.0093^{\ast}$ $[0.0044,0.0147]$	$0.0112^{\ast}$ $[0.0057,0.0175]$	$0.0087^{\ast}$ $[0.0026,0.0151]$

Table 2: Three separate matched CIFAR packages. The first row uses the two conditioning runs of correct source

+

transport; the other rows compare transport structures. Each evaluated transport route uses

J=0

. Signs follow the first column; entries show mean [95% paired class-cluster CI] and stars use unrounded intervals. One training seed per arm.

Panel of Figure 5 — Figure 5: CIFAR target and source budgets. Left: final-test SW for correct source $+$ transport and target-adapted baselines. The main route uses no target support at any $K_{T}$ ; its small changes across budgets reflect different sampled source/query episodes and generation noise. Right: $K_{T}=1$ with nested source supports and the checkpoint trained at $M_{S}=64$ . Error bars show marginal class-cluster percentile 95% intervals; paired effects and stars are computed separately.

Panel of Figure 6 — Figure 6: Office-Home exposes a different boundary from CIFAR. Left: correct-source transport remains nearly flat in DINO feature-SW, while full fine-tuning improves as target support grows. Right: generated routes can become strongly Clipart-like without preserving the requested object; the star is the held-out real Clipart audit. The natural-domain limitation is therefore semantic, not simply a failure to change domain appearance.

Setting	Representation / transport evidence	Generative implication
GMM	Bayes-ID can be perfect while learned-ID is weaker; at $\alpha=1$ , learned-ID $.883$ and transported retrieval $.315$ . Shift sweep: transport-over-reuse $.024\!\rightarrow\!.541$ as headroom grows.	Correct-vs-wrong SW gain $.523$ at $(1,0)$ ; usable identity tracks the gain more closely than Bayes-ID
CIFAR	Transported target-coordinate retrieval $49$ – $62.5\%$ across relations	Correct source $+$ transport improves on wrong/relation-only at $K_{T}=0$ ; the separate final-test budget comparison is in Table 1
Office-Home	Product- $z$ retrieval $91.0\%$ ; transport raises target-coordinate retrieval $12.0\%\!\rightarrow\!42.0\%$	Clipart share $86.5\%$ ; low object ID persists even with abundant direct $z_{T}$ , showing that transport error alone is insufficient to explain the failure.

Table 3: Stages of source-informed transfer. Representative diagnostics identify conditions enabling transfer and the stage that becomes limiting; values are not intended as a common metric across rows.

Appendix figures & tables52 assets

Supplementary material from the paper’s appendix.

Appendix

Task family	Own	Mean	$\Delta_{\rm task}$
Perturbation .10	$0.4398$	$0.5956$	$0.1558\pm 0.0286^{\ast}$
Perturbation .25	$0.4912$	$0.6125$	$0.1212\pm 0.0232^{\ast}$
Perturbation .50	$0.5847$	$0.6729$	$0.0882\pm 0.0272^{\ast}$
Perturbation 1.00	$0.6684$	$0.7591$	$0.0907\pm 0.0423^{\ast}$
Unrelated	$0.5427$	$0.6386$	$0.0959\pm 0.0341^{\ast}$

Appendix

Table 4: GMM calibration on five separate 80k-step checkpoints. Each uses 32 held-out tasks; eight noise draws are averaged within task.

\Delta_{\rm task}=L(\bar{z})-L(z_{y})

has a paired task-level 95% half-width. This is a coordinate intervention, not a deployment comparison.

Figure 7: Task-coordinate interpolation. The GMM and CIFAR panels use their respective model checkpoints. Losses are normalised by the shared-coordinate loss; well depth is the percentage drop from this loss to the minimum along the own-task direction. Own-task and wrong-task directions keep their diagnostic meanings. Their shapes do not replace the paired deployment-route comparisons.

$\alpha\backslash\eta$	0	0.1	0.25	0.5	1	2
0	0.000	0.000	-0.000	0.000	-0.000	-0.002
0.1	0.008	0.010	0.006	0.006	0.002	0.002
0.25	0.093	0.102	0.098	0.066	0.038	0.023
0.5	0.242	0.209	0.205	0.220	0.140	0.091
0.75	0.387	0.390	0.370	0.321	0.256	0.142
1	0.523	0.563	0.523	0.447	0.404	0.228

Appendix

Table 5: Complete GMM factorial diagnostics. Top: correct-source advantage

G_{\rm source}=\mathrm{SW}_{wrong}-\mathrm{SW}_{correct}

. Middle: learned-coordinate task-ID accuracy. Bottom: held-out oracle affine relation

R^{2}

. Each cell averages five seeds; generation effects further average 64 held-out tasks per seed.

Figure 8: Relation predictability is a separate boundary from task identifiability. Each colour fixes source-task separation $\alpha$ while $\eta$ varies. The horizontal axis is the held-out $R^{2}$ of an affine map fitted to true training-task component means; it is therefore independent of Meta-Diff’s learned coordinate. At $\alpha=1$ , Bayes-ID remains 1.0 while relation $R^{2}$ falls from 1.000 to .408 and the correct-over-wrong SW advantage falls from .523 to .228. Over the full grid, $\rho(G_{\rm source},R^{2}_{\rm rel})=.54$ .

Panel of Figure 9 — Figure 9: Full-grid GMM controls. Left: relation-only minus correct-source SW; positive values favour the source-conditioned route and stars mark hierarchical paired intervals excluding zero. Right: true-source pairwise SW (in units of $10^{-3}$ ), an outcome-independent source-separation diagnostic. The source-conditioned route has a resolved advantage over relation-only in 33 of 42 cells; Table 7 also contains 34 resolved correct-versus-wrong cells, while source-reuse and abundant-target-reference comparisons are not uniformly favourable.

$\eta$	Correct	Wrong	Relation-only	Source reuse	Abundant-target reference
0	0.09386	0.09396	0.09361	0.64158	0.09311
0.1	0.14144	0.14170	0.14129	0.70917	0.14840
0.25	0.16930	0.16911	0.16933	0.63886	0.16775
0.5	0.32266	0.32270	0.32317	0.77757	0.31470
1	0.58670	0.58635	0.58467	0.71809	0.52292
2	1.04219	1.04036	1.04108	1.16237	0.90496

Appendix

Table 6: Complete factorial GMM SW route means. Lower is better; five seeds and 64 held-out tasks per seed.

$\eta$	Wrong $-$ correct	Relation-only $-$ correct
0	$+0.00010\,[-0.00035,+0.00059]$	$-0.00025\,[-0.00086,+0.00036]$
0.1	$+0.00025\,[-0.00054,+0.00093]$	$-0.00015\,[-0.00087,+0.00044]$
0.25	$-0.00019\,[-0.00106,+0.00062]$	$+0.00003\,[-0.00105,+0.00102]$
0.5	$+0.00004\,[-0.00138,+0.00160]$	$+0.00051\,[-0.00113,+0.00227]$
1	$-0.00035\,[-0.00319,+0.00254]$	$-0.00203\,[-0.00437,+0.00043]$
2	$-0.00182\,[-0.00695,+0.00300]$	$-0.00110\,[-0.00633,+0.00385]$

Appendix

Table 7: Complete factorial GMM SW source controls: comparator minus correct-source transport, with hierarchical seed/task 95% intervals. Positive favours correct-source transport.

$\eta$	Source reuse $-$ correct	Abundant-target ref. $-$ correct
0	$+0.54772^{\ast}\,[+0.47915,+0.61436]$	$-0.00075\,[-0.02123,+0.01295]$
0.1	$+0.56772^{\ast}\,[+0.50726,+0.63460]$	$+0.00695\,[-0.00126,+0.01545]$
0.25	$+0.46955^{\ast}\,[+0.40729,+0.51238]$	$-0.00156\,[-0.01588,+0.01273]$
0.5	$+0.45492^{\ast}\,[+0.37794,+0.53592]$	$-0.00796\,[-0.02093,+0.00691]$
1	$+0.13139\,[-0.01521,+0.26379]$	$-0.06379^{\ast}\,[-0.09976,-0.02633]$
2	$+0.12019\,[-0.02686,+0.29146]$	$-0.13723^{\ast}\,[-0.18348,-0.08521]$

Appendix

Table 8: Complete factorial GMM SW reuse/reference contrasts: comparator minus correct-source transport, with hierarchical seed/task 95% intervals. Positive favours correct-source transport. The abundant-target coordinate reference encodes 512 target samples with the frozen encoder and uses the same frozen generator, without target-time coordinate optimisation. It is a reference route, not a performance upper bound.

$\eta$	Correct	Wrong	Relation-only	Source reuse	Abundant-target reference
0	0.01827	0.01820	0.01813	0.88275	0.01782
0.1	0.03767	0.03772	0.03767	1.01943	0.04387
0.25	0.06638	0.06640	0.06653	0.87212	0.06448
0.5	0.14388	0.14391	0.14401	1.02813	0.13898
1	0.32266	0.32176	0.32035	0.71514	0.24480
2	0.72649	0.72400	0.72402	1.19824	0.52956

Appendix

Table 9: Complete factorial GMM MMD route means. Lower is better; five seeds and 64 held-out tasks per seed.

$\eta$	Wrong $-$ correct	Relation-only $-$ correct
0	$-0.00007\,[-0.00025,+0.00007]$	$-0.00014\,[-0.00033,+0.00003]$
0.1	$+0.00005\,[-0.00008,+0.00018]$	$+0.00000\,[-0.00013,+0.00012]$
0.25	$+0.00003\,[-0.00022,+0.00026]$	$+0.00015\,[-0.00009,+0.00044]$
0.5	$+0.00004\,[-0.00072,+0.00090]$	$+0.00014\,[-0.00066,+0.00098]$
1	$-0.00090\,[-0.00334,+0.00176]$	$-0.00231^{\ast}\,[-0.00389,-0.00061]$
2	$-0.00249\,[-0.00761,+0.00164]$	$-0.00247\,[-0.00659,+0.00100]$

Appendix

Table 10: Complete factorial GMM MMD source controls: comparator minus correct-source transport, with hierarchical seed/task 95% intervals. Positive favours correct-source transport.

$\eta$	Source reuse $-$ correct	Abundant-target ref. $-$ correct
0	$+0.86448^{\ast}\,[+0.62391,+1.08397]$	$-0.00044\,[-0.01105,+0.00643]$
0.1	$+0.98176^{\ast}\,[+0.86073,+1.10203]$	$+0.00620^{\ast}\,[+0.00008,+0.01270]$
0.25	$+0.80574^{\ast}\,[+0.62634,+0.92682]$	$-0.00190\,[-0.01245,+0.00877]$
0.5	$+0.88426^{\ast}\,[+0.69128,+1.05914]$	$-0.00489\,[-0.02169,+0.01614]$
1	$+0.39248^{\ast}\,[+0.18649,+0.59469]$	$-0.07786^{\ast}\,[-0.11287,-0.04096]$
2	$+0.47175^{\ast}\,[+0.24454,+0.74321]$	$-0.19693^{\ast}\,[-0.24489,-0.15161]$

Appendix

Table 11: Complete factorial GMM MMD reuse/reference contrasts: comparator minus correct-source transport, with hierarchical seed/task 95% intervals. Positive favours correct-source transport. The abundant-target coordinate reference encodes 512 target samples with the frozen encoder and uses the same frozen generator, without target-time coordinate optimisation. It is a reference route, not a performance upper bound.

$\alpha$	Bayes-ID	learned-ID	Collapse gap
0.00	.016	.016	$-.001$
0.10	.400	.073	.327
0.25	.953	.325	.628
0.50	1.000	.708	.292
0.75	1.000	.839	.161
1.00	1.000	.874	.126

Appendix

Table 12: Intrinsic versus usable source-task identifiability. Chance is

1/64=.0156

. The collapse gap is Bayes-ID minus learned-ID.

Predictors	Fit $R^{2}$	Key coefficient	95% CI
Bayes-ID $+R^{2}_{\rm rel}+$ interaction	.587	$I_{\rm Bayes}\!\times R^{2}_{\rm rel}=+.54$	$[+.17,+.92]$
learned-ID $+R^{2}_{\rm rel}+$ interaction	.843	$I_{\rm learned}\!\times R^{2}_{\rm rel}=+.61$	$[+.33,+.89]$
Bayes-ID $+$ learned-ID $+R^{2}_{\rm rel}$	.814	$I_{\rm learned}=+.73$	$[+.55,+.91]$
		$I_{\rm Bayes}=-.27$	$[-.44,-.10]$

Appendix

Table 13: Descriptive GMM regressions predicting

G_{\rm source}

. OLS fits use the 42 unrounded seed-mean cells. Displayed coefficient intervals are the conventional homoskedastic normal approximation

\widehat{\beta}\pm 1.96\,\mathrm{SE}_{\rm OLS}

with residual variance estimated using

42-p

degrees of freedom (

p=4

including the intercept). These descriptive cell-level intervals do not account for matched-seed dependence across cells and are not the hierarchical seed/task intervals for route comparisons.

$\alpha$	Bayes-ID	learned $z_{S}$	transported $\widetilde{z}_{T}$	raw $z_{S}\!\to z_{T}$
0.00	.016	.015	.013	.016
0.10	.400	.071	.019	.015
0.25	.953	.323	.089	.023
0.50	1.000	.691	.169	.039
0.75	1.000	.831	.258	.074
1.00	1.000	.883	.315	.109

Appendix

Table 14: Mean top-1 task identity through the GMM pipeline, averaged over relation levels and seeds. Chance is

1/64=.0156

$\lambda$	Correct	Wrong	Relation-only	Source reuse	$G_{\rm source}$	Transport over reuse
0.00	.759	1.241	1.062	.783	+.482	+.024
0.25	.765	1.252	1.048	.828	+.487	+.063
0.50	.802	1.315	1.115	.905	+.513	+.102
1.00	.835	1.355	1.157	1.066	+.520	+.231
2.00	1.037	1.667	1.443	1.577	+.631	+.541

Appendix

Table 15: GMM shift-magnitude sweep: route means averaged over tasks and five seeds.

G_{\rm source}=\mathrm{SW}_{\rm wrong}-\mathrm{SW}_{\rm correct}

; transport-over-reuse is

\mathrm{SW}_{\rm reuse}-\mathrm{SW}_{\rm correct}

Configuration	No adapt.	Oracle	Full FT ref.	Captured
$k=16$ , 80k	.8821	.6320	.1276	33.2%
$k=16$ , 320k	.8741	.5965	.1508	38.4%
$k=64$ , 80k	.8868	.5396	.1267	45.7%
$k=64$ , 320k	.8779	.4690	.1565	56.7%

Appendix

Table 16: Historical GMM representation-capacity diagnostic. The oracle fits an abundant-target coordinate; captured fraction is

(e_{0}-e_{\rm oracle})/(e_{0}-e_{\rm FT})

Task family	No adapt.	Oracle	Best-grid FT	Captured
Perturbation .10	.7513	.3835	.1179	58.1%
Perturbation .25	.8910	.5583	.1220	43.3%
Perturbation .50	.9516	.6949	.1359	31.5%
Perturbation 1.00	.9679	.7808	.1239	22.2%

Appendix

Table 17: Historical GMM task-relatedness diagnostic at

k=16

, 80k steps. Larger template perturbations produce less related tasks.

Route / comparison	$K_{T}=1$	$K_{T}=5$	$K_{T}=20$
correct source $+$ transport	$0.2071$ $[0.1797,0.2421]$	$0.2077$ $[0.1831,0.2380]$	$0.2129$ $[0.1854,0.2471]$
Source-init. adaptation	$0.2436$ $[0.2186,0.2756]$	$0.2351$ $[0.2108,0.2652]$	$0.2203$ $[0.1901,0.2597]$
Target-only	$0.2255$ $[0.1986,0.2605]$	$0.2130$ $[0.1858,0.2469]$	$0.2087$ $[0.1781,0.2481]$
Mean source	$0.2309$ $[0.2019,0.2667]$	$0.2347$ $[0.2073,0.2663]$	$0.2379$ $[0.2080,0.2731]$
Shuffled source	$0.2417$ $[0.2113,0.2779]$	$0.2453$ $[0.2165,0.2774]$	$0.2469$ $[0.2155,0.2827]$
Relation-only	$0.2413$ $[0.2126,0.2764]$	$0.2427$ $[0.2158,0.2739]$	$0.2476$ $[0.2183,0.2818]$

Appendix

Table 18: CIFAR validation source-dependence sweep: 180 episodes, 60 per budget. Main transport and mean/shuffled/relation controls use

J=0

. Source-initialised adaptation uses target support and is not strict reuse. SW is lower-is-better; paired

\Delta

is comparator minus main. Entries show mean [95% class-cluster CI] with unrounded paired stars.

Figure 10: CIFAR $K_{T}=1$ historical deployment grid from the 50k-step checkpoint. The seven rows show no adaptation, source-initialised target adaptation ( $J=25$ ), correct source $+$ transport ( $J=0$ ), transport plus refinement, target-only, a query-assisted target-coordinate reference, and real target. The source-initialised route uses target support and is not strict reuse. The query-assisted reference starts at the transported coordinate and optimises the coordinate on the same target-query set used as its reference, without a coordinate prior, for 25 steps; its frozen generator is unchanged. It is a privileged diagnostic, not a deployable baseline or a performance upper bound. Generated rows retain the original samples and paired noise; the bottom row is real target data.

Target budget	Full-FT steps	Learning rate
$K_{T}=1$	25	$10^{-5}$
$K_{T}=5$	25	$10^{-4}$
$K_{T}=20$	400	$10^{-5}$

Appendix

Table 19: CIFAR validation-selected full-fine-tuning settings frozen before the matched final-test evaluation.

$K_{T}$	Refinement gain [95% CI]
1	$-0.0244\,[-0.0365,-0.0120]^{\ast}$
2	$-0.0071\,[-0.0249,0.0116]$
5	$-0.0267\,[-0.0468,-0.0061]^{\ast}$
10	$0.0144\,[-0.0169,0.0459]$
20	$-0.0065\,[-0.0511,0.0380]$

Appendix

Table 20: Matched refinement ablation on the correct source

+

transport checkpoint.

J=0

remains the main route; the ablation uses

J=25

, learning rate

0.01

and the same prior/noise budget. Entries report paired 95% class-cluster intervals; stars use unrounded gain intervals.

Configuration / comparison	$K_{T}=1$	$K_{T}=5$	$K_{T}=20$
Basis: correct source $+$ transport	$0.2067$ $[0.1793,0.2414]$	$0.2090$ $[0.1834,0.2391]$	$0.2136$ $[0.1864,0.2469]$
FiLM: correct source $+$ transport	$0.1973$ $[0.1689,0.2355]$	$0.1942$ $[0.1709,0.2239]$	$0.1987$ $[0.1720,0.2334]$
FiLM $-$ basis	$-0.0094$ $[-0.0238,0.0046]$	$-0.0148$ $[-0.0298,0.0003]$	$-0.0149$ $[-0.0300,0.0003]$
Basis residual MLP	$0.2102$ $[0.1839,0.2451]$	$0.2109$ $[0.1879,0.2392]$	$0.2138$ $[0.1870,0.2481]$
Basis linear	$0.2124$ $[0.1829,0.2516]$	$0.2113$ $[0.1859,0.2442]$	$0.2174$ $[0.1886,0.2550]$
Linear $-$ MLP	$0.0022$ $[-0.0031,0.0080]$	$0.0004$ $[-0.0056,0.0068]$	$0.0036$ $[-0.0019,0.0097]$

Appendix

Table 21: Three independent matched CIFAR SW packages. Upper block: correct source

+

transport with two conditioning architectures. Lower blocks: transport-structure ablations, with direct

J=0

generation in both compared arms. Pairing is within block only; intervals resample validation classes at one training seed per arm.

Group	Relation	Metric	Raw $z_{S}$	Main	Wrong-source + transport	Relation only
Seen	Blur	L2	$36.50/-0.01050$	$71.50/+0.02241$	$0.42/-0.24324$	$1.67/-0.13592$
		Cosine	$26.58/-0.06849$	$72.58/+0.01675$	$0.42/-0.24228$	$1.67/-0.10330$
Seen	Noise	L2	$56.83/+0.00548$	$74.50/+0.02872$	$0.33/-0.26493$	$1.67/-0.13956$
		Cosine	$30.50/-0.04843$	$78.08/+0.01791$	$0.25/-0.24565$	$1.67/-0.09906$
Seen	Contrast	L2	$36.00/-0.01447$	$68.83/+0.02719$	$0.50/-0.26657$	$1.67/-0.12781$
		Cosine	$25.58/-0.07178$	$72.08/+0.02048$	$0.67/-0.28837$	$1.67/-0.11249$

Appendix

Table 22: Complete CIFAR frozen-coordinate retrieval. Each cell is top-1 percentage / mean nearest-wrong margin. Positive margins favour the requested class; relation-only chance/tie behaviour is not task discrimination.

Group	Relation	Main $-$ raw source (pp) [95% CI]	Main $-$ wrong source (pp) [95% CI]
Seen	Blur	$+35.00\,[+32.25,+37.67]^{\ast}$	$+71.08\,[+69.08,+73.08]^{\ast}$
Seen	Noise	$+17.67\,[+15.33,+20.00]^{\ast}$	$+74.17\,[+72.25,+75.92]^{\ast}$
Seen	Contrast	$+32.83\,[+30.67,+35.08]^{\ast}$	$+68.33\,[+66.58,+70.08]^{\ast}$
Held-out	Blur	$+45.25\,[+42.25,+48.25]^{\ast}$	$+61.25\,[+58.25,+64.00]^{\ast}$
Held-out	Noise	$+33.00\,[+27.25,+38.25]^{\ast}$	$+58.75\,[+54.50,+63.25]^{\ast}$
Held-out	Contrast	$+35.00\,[+30.25,+39.50]^{\ast}$	$+46.75\,[+43.00,+50.50]^{\ast}$

Appendix

Table 23: CIFAR L2 retrieval gains in percentage points, main minus comparator. Intervals bootstrap the 20 paired support resamples conditional on the fixed class set; positive favours correct-source transport.

Group	Relation	Generated identity (%)	Real identity (%)	DINO-SW $\downarrow$	DINO-MMD $\downarrow$
Seen	Blur	5.52	50.80	0.02176	0.15937
Seen	Noise	3.78	29.27	0.01671	0.09777
Seen	Contrast	6.84	78.10	0.02983	0.27962
Held-out	Blur	3.59	51.20	0.02338	0.18884
Held-out	Noise	3.85	34.50	0.01888	0.13402
Held-out	Contrast	2.71	70.90	0.03059	0.28967

Appendix

Table 24: CIFAR direct-target generation using abundant target support. Identity rates are percentages; SW and MMD use the diagnostic DINO feature space, not the headline pixel-space metrics.

Relation	Identity $\Delta$ (pp) [95% CI]
Blur	$-1.93\,[-6.70,+4.11]$
Noise	$+0.07\,[-4.34,+6.18]$
Contrast	$-4.13\,[-7.99,+0.14]$

Appendix

Table 25: CIFAR direct-target generated-identity differences, held-out minus seen, in percentage points. The two different class groups are independently resampled; these are descriptive group contrasts, not paired causal effects.

Group	Relation	Timesteps	Correct MSE	Wrong MSE	$\Delta_{z}$ [95% CI]
Seen	Blur	0–199	0.069118	0.069843	$+0.00073\,[+0.00058,+0.00089]^{\ast}$
		200–399	0.025385	0.026199	$+0.00081\,[+0.00065,+0.00099]^{\ast}$
		400–599	0.014950	0.015695	$+0.00074\,[+0.00061,+0.00089]^{\ast}$
		600–799	0.008320	0.008864	$+0.00054\,[+0.00045,+0.00065]^{\ast}$
		800–999	0.002333	0.002566	$+0.00023\,[+0.00019,+0.00028]^{\ast}$
Seen	Noise	0–199	0.575592	0.576098	$+0.00051\,[+0.00040,+0.00062]^{\ast}$

Appendix

Table 26: CIFAR coordinate use across diffusion time: absolute denoising MSE and wrong-minus-correct differences with class-bootstrap 95% intervals.

Check	Result
CLIP top-1, Product / Clipart	89.03% / 65.96%
Test classes with Clipart top-1 $\geq 30\%$	10/10
DINOv2 held-out domain accuracy	98.30%
Same-class centroid top-1 / median rank	100.0% / 1
Same-minus-different centroid cosine gap	.5077

Appendix

Table 27: Office-Home real-image evaluator audit.

Checkpoint step	Main $K_{T}=0$ DINO-SW
20,000	.040525
25,000	.039609
30,000	.038712
35,000	.038019
40,000	.037572
45,000	.037351

Appendix

Table 28: Office-Home validation-only selection record. The upper block reports the main-route checkpoint trajectory. The lower block reports the budget-specific target-only and full-fine-tuning settings subsequently frozen for final testing. These validation values are not pooled with test results.

Route	DINO-SW	MMD	Object ID (%)	Clipart (%)
Correct $+$ transport	.038969	.407846	1.67	86.50
Wrong $+$ transport	.039394	.419016	1.44	86.27
Relation-only	.039251	.415280	1.37	86.71
Source reuse	.041641	.457524	1.18	63.80

Appendix

Table 29: Office-Home

K_{T}=0

route matrix. Distance metrics are lower-is-better; requested-class and Clipart fractions are higher-is-better.

$K_{T}$	Route	DINO-SW	MMD	Object ID (%)	Clipart (%)
1	Correct $+$ transport	0.038969	0.407846	1.67	86.50
1	Target-only	0.040346	0.439860	2.23	75.73
1	Full FT	0.039336	0.468223	5.33	87.49
5	Correct $+$ transport	0.038969	0.407846	1.67	86.50
5	Target-only	0.039031	0.409734	1.81	81.22
5	Full FT	0.034550	0.349838	12.55	83.11

Appendix

Table 30: Office-Home target-budget route means. The same zero-target correct-source generations are reused across budgets; adapting baselines receive the stated target support. SW and MMD are lower-is-better; identity and Clipart rates are percentages.

Figure 11: Office-Home generation grid. Rows are held-out classes. Columns contain real Product, real Clipart, source reuse, relation-only, wrong-source transport, correct-source transport, one-shot target-only and one-shot full fine-tuning. Generated routes share DDIM noise within class.

$K_{T}$	Comparator	$\Delta$ DINO-SW [95% CI]	$\Delta$ MMD [95% CI]
0	Wrong + transport	$0.000425^{*}$ [0.000213, 0.000643]	$0.011170^{*}$ [0.006870, 0.015706]
0	Relation-only	$0.000282^{*}$ [0.000109, 0.000456]	$0.007434^{*}$ [0.002360, 0.012075]
0	Source reuse	$0.002673^{*}$ [0.002270, 0.003036]	$0.049678^{*}$ [0.039741, 0.060430]
1	Target-only	$0.001377^{*}$ [0.000568, 0.002311]	$0.032014^{*}$ [0.015413, 0.047276]
1	Full FT	$0.000367$ [-0.001578, 0.002312]	$0.060376^{*}$ [0.019137, 0.101637]
5	Target-only	$0.000062$ [-0.000754, 0.000800]	$0.001888$ [-0.019453, 0.018779]

Appendix

Table 31: Office-Home paired distributional gains with 95% class-bootstrap intervals. Positive favours correct-source transport; negative favours the comparator.

$K_{T}$	Comparator	$\Delta$ object ID [95% CI]	$\Delta$ Clipart [95% CI]
0	Wrong + transport	$0.230$ [-0.043, 0.543]	$0.226$ [-0.438, 0.955]
0	Relation-only	$0.299$ [-0.078, 0.885]	$-0.208$ [-0.885, 0.443]
0	Source reuse	$0.482$ [-0.221, 1.406]	$22.695^{*}$ [20.820, 24.661]
1	Target-only	$-0.560$ [-2.031, 0.404]	$10.768^{*}$ [2.760, 19.062]
1	Full FT	$-3.659$ [-8.737, 0.195]	$-0.990$ [-4.844, 2.734]
5	Target-only	$-0.143$ [-0.703, 0.286]	$5.273^{*}$ [1.667, 9.532]

Appendix

Table 32: Office-Home paired identity and domain-share gains with 95% class-bootstrap intervals, in percentage points. Positive favours correct-source transport.

Query route	L2 top-1	Cosine top-1	L2 margin	Cosine margin
Raw $z_{S}$	12.0%	26.5%	-.0500	-.1749
Correct source $+$ transport	42.0%	44.5%	-.0153	-.0620
Wrong-source transport	20.0%	19.0%	-.0897	-.2654
Relation-only	10.0%	10.0%	-.0545	-.1510

Appendix

Table 33: Office-Home target-coordinate retrieval on ten held-out classes.

Comparator	L2 $\Delta$ [95% CI]	Cosine $\Delta$ [95% CI]
Raw $z_{S}$	$+30.00\,[+4.00,+58.50]^{\ast}$	$+18.00\,[-2.50,+43.50]$
Wrong-source transport	$+22.00\,[-11.50,+56.00]$	$+25.50\,[-10.00,+60.00]$
Relation only	$+32.00\,[+5.50,+59.50]^{\ast}$	$+34.50\,[+7.00,+61.50]^{\ast}$

Appendix

Table 34: Office-Home target-coordinate retrieval gains, main minus comparator, in percentage points with paired class-bootstrap 95% intervals.

Group	Generated ID	Real ID	DINO-SW	DINO-MMD	Clipart
Seen (45 classes)	2.38%	50.33%	.03679	.35175	87.34%
Held-out (10 classes)	1.56%	52.00%	.03902	.40642	88.33%

Appendix

Table 35: Office-Home direct-

z_{T}

generation using abundant non-query Clipart support.

Group	Timesteps	Correct MSE	Wrong MSE	$\Delta_{z}$ [95% CI]
Seen	0–199	0.056513	0.056724	$+0.000211\,[+0.000141,+0.000294]^{\ast}$
	200–399	0.019655	0.019785	$+0.000131\,[+0.000086,+0.000183]^{\ast}$
	400–599	0.013028	0.013164	$+0.000136\,[+0.000089,+0.000189]^{\ast}$
	600–799	0.008825	0.008963	$+0.000138\,[+0.000095,+0.000187]^{\ast}$
	800–999	0.003287	0.003342	$+0.000055\,[+0.000039,+0.000072]^{\ast}$
Held-out	0–199	0.059910	0.059928	$+0.000018\,[-0.000044,+0.000073]$

Appendix

Table 36: Office-Home direct-target coordinate use across diffusion time. MSEs use identical query/noise/timestep pairs; positive wrong-minus-correct differences favour the correct coordinate. Intervals resample classes.

Seen-class route	Generated identity (%)	Real identity (%)	DINO-SW $\downarrow$	DINO-MMD $\downarrow$	Clipart (%)
Original direct $z_{T}$	2.38	50.33	0.03679	0.35175	87.34
Explicit class embedding	6.90	50.33	0.03507	0.32768	87.45

Appendix

Table 37: Office-Home separately trained class-conditioning capacity control on 45 seen classes. Identity and Clipart columns are percentages.

Metric	Favor-control $\Delta$ [95% CI]
Requested-class identity	$+4.51\,\mathrm{pp}\,[+2.85,+6.37]^{\ast}$
DINO-SW reduction	$+0.00172\,[+0.00121,+0.00229]^{\ast}$
DINO-MMD reduction	$+0.02407\,[+0.01332,+0.03671]^{\ast}$
Clipart-share increase	$+0.12\,\mathrm{pp}\,[-1.30,+1.53]$

Appendix

Table 38: Office-Home capacity-control effects with 95% intervals paired over 45 classes. Identity/domain gains are control minus direct-

z_{T}

; SW/MMD reductions are direct-

z_{T}

minus control.

Method	$K_{T}=1$	$2$	$5$	$10$	$20$
No adaptation	.3696	.3696	.3696	.3696	.3696
Target-only	.1397	.1406	.1398	.1405	.1408
Relation-only	.3680	.3680	.3680	.3680	.3680
Wrong-source transport	.1407	.1407	.1407	.1407	.1407
Correct source $+$ transport	.1407	.1407	.1407	.1407	.1407
Full fine-tuning	.2981	.2802	.2304	.1800	.1591

Appendix

Table 39: ChestX-ray14 SW route means. Lower is better. The independently reported wrong-minus-correct paired interval at

K_{T}=1

7.7\times 10^{-6}\pm 2.3\times 10^{-5}

; other displayed route differences lack paired covariance information.

Protocol	Route	Statistic	SW	MMD
Disease IV–VI	Source reuse	2.5248	.3642	7.1765
	Wrong disease, same family	2.5020	.3638	7.1466
	Wrong disease, different family	2.5025	.3639	7.1444
	Correct source $+$ transport	2.5030	.3638	7.1429
Family IV–VI	Source reuse	1.7452	.3081	5.7946
	Wrong-family transport	1.7185	.3064	5.7468

Appendix

Table 40: Fitzpatrick zero-target route means. Invalidated phenotype/semantic instruments are omitted here; statistic distance, SW and energy MMD are lower-is-better.

Check	Result
Target tissue balanced accuracy	88.25%
Minimum target tissue recall	60.57%
Domain balanced accuracy, fold 1	59.01%
Domain balanced accuracy, fold 2	58.24%
Domain balanced accuracy, fold 3	57.00%
Same-tissue centroid retrieval	100% top-1 in each fold

Appendix

Table 41: CRC real-image preflight and stopping decision.

Route	SW	Energy MMD	DINO-100 (%)
Correct source $+$ transport	$0.1781$ $[0.1627,0.1959]$	$2.3306$ $[1.9624,2.7620]$	$2.53$ $[0.54,5.73]$
Clean-source generation $+$ corruption	$0.1688$ $[0.1528,0.1867]$	$2.3336$ $[1.9307,2.8087]$	$2.62$ $[0.57,5.71]$
Paired effect	$-0.0092^{\ast}$ $[-0.0138,-0.0045]$	$0.0030$ $[-0.0748,0.0827]$	$-0.09$ $[-0.59,0.35]$

Appendix

Table 42: Known-corruption baseline on 60 matched test episodes. Entries are mean [95% class-cluster CI]. For SW/MMD, the paired effect is image-space baseline minus transport; for identity, it is transport minus image-space baseline in percentage points. Positive favours transport; stars mark paired intervals excluding zero.

Updates	Learning rate	Mean validation SW
25	$10^{-5}$	0.218506
25	$10^{-4}$	0.259794
100	$10^{-5}$	0.217296
100	$10^{-4}$	0.310748

Appendix

Table 43: Validation selection for the common full-FT schedule. Lower mean SW is better; no test outcome selects the setting.

Route and metric	$K_{T}=1$	$K_{T}=5$	$K_{T}=20$
Transport: SW	$0.1828$ $[0.1634,0.2036]$	$0.1828$ $[0.1634,0.2036]$	$0.1828$ $[0.1634,0.2036]$
Transport: MMD	$2.5036$ $[2.0193,3.0320]$	$2.5036$ $[2.0193,3.0320]$	$2.5036$ $[2.0193,3.0320]$
Transport: DINO-100 (%)	$2.64$ $[0.40,5.97]$	$2.64$ $[0.40,5.97]$	$2.64$ $[0.40,5.97]$
Target-only: SW	$0.2108$ $[0.1857,0.2381]$	$0.1905$ $[0.1690,0.2138]$	$0.1774$ $[0.1615,0.1937]$
Target-only: MMD	$3.5534$ $[2.7894,4.4358]$	$2.7992$ $[2.2504,3.3998]$	$2.4628$ $[2.0898,2.8443]$
Target-only: DINO-100 (%)	$2.64$ $[0.50,5.99]$	$2.74$ $[0.47,6.18]$	$2.69$ $[0.52,5.95]$

Appendix

Table 44: Controlled-budget test panel, with 60 matched episodes per budget. Mean [95% class-cluster CI]. The full-FT update schedule is constant across columns; zero-target transport has identical samples across columns.

Comparison	$K_{T}=1$ / change	$K_{T}=5$	$K_{T}=20$
Target-only $-$ transport: SW	$0.0280^{\ast}$ $[0.0127,0.0448]$	$0.0077^{\ast}$ $[0.0021,0.0134]$	$-0.0054$ $[-0.0121,0.0014]$
Full FT $-$ transport: SW	$0.1226^{\ast}$ $[0.0899,0.1568]$	$0.0294^{\ast}$ $[0.0033,0.0570]$	$-0.0119$ $[-0.0429,0.0213]$
Full FT: $K_{T}=5$ minus 1	$-0.0932^{\ast}$ $[-0.1270,-0.0564]$	–	–
Full FT: $K_{T}=20$ minus 1	$-0.1345^{\ast}$ $[-0.1730,-0.0957]$	–	–

Appendix

Table 45: Paired SW effects for the controlled-budget panel. The first two rows compare routes at each budget; positive favours transport. The last two rows report changes in full-FT SW as

K_{T}

increases; negative favours the larger support set.

Figure 12: Controlled full-FT update schedule across target budgets. Error bars are marginal 95% class-cluster intervals; route and support-size comparisons use the paired intervals in Table 45 .

Route / paired effect	$K_{T}=1$	$K_{T}=5$	$K_{T}=20$
Transport	$2.53$ $[0.54,5.73]$	$2.85$ $[0.54,6.46]$	$2.34$ $[0.35,5.49]$
Target-only	$2.38$ $[0.36,5.62]$	$2.71$ $[0.45,6.25]$	$2.50$ $[0.49,5.54]$
Full FT	$3.68$ $[1.02,7.20]$	$9.58$ $[3.85,16.82]$	$10.50$ $[5.35,16.70]$
Real-query calibration	$50.49$ $[42.20,58.79]$	$49.79$ $[41.59,57.99]$	$50.65$ $[42.23,59.13]$
Transport $-$ target-only	$0.16$ $[-0.14,0.49]$	$0.14$ $[-0.07,0.33]$	$-0.16$ $[-0.45,0.09]$
Transport $-$ full FT	$-1.15^{\ast}$ $[-2.67,-0.09]$	$-6.74^{\ast}$ $[-11.79,-3.00]$	$-8.16^{\ast}$ $[-12.00,-4.67]$

Appendix

Table 46: Matched DINO-100 requested-class fractions (%) and real-query calibration, with 95% class-cluster intervals. Paired effects are percentage points; positive favours transport.

Metric / paired effect	$K_{T}=1$	$K_{T}=5$	$K_{T}=20$
Full FT: SW	$0.2649$ $[0.2382,0.2927]$	$0.2177$ $[0.2002,0.2355]$	$0.1487$ $[0.1395,0.1591]$
Full FT $-$ transport: SW	$0.0868^{\ast}$ $[0.0625,0.1107]$	$0.0343^{\ast}$ $[0.0122,0.0577]$	$-0.0357^{\ast}$ $[-0.0586,-0.0146]$
Full FT: energy MMD	$5.9003$ $[4.7552,7.1604]$	$3.9034$ $[3.2829,4.5988]$	$1.9151$ $[1.7299,2.1178]$
Full FT $-$ transport: MMD	$3.5698^{\ast}$ $[2.5478,4.6246]$	$1.5648^{\ast}$ $[0.8988,2.2642]$	$-0.4890$ $[-1.0680,0.0466]$

Appendix

Table 47: SW and energy MMD for the new seeded full-FT replication used for semantic evaluation. Table 1 B retains the earlier budget-specific comparison.

Route / paired effect	$K_{T}=1$	$K_{T}=5$	$K_{T}=10$
Transport	$1.67$ $[0.60,3.19]$	$1.67$ $[0.60,3.19]$	$1.67$ $[0.60,3.19]$
Target-only	$2.23$ $[0.52,5.01]$	$1.81$ $[0.65,3.35]$	$1.61$ $[0.59,2.94]$
Full FT	$5.33$ $[0.63,11.59]$	$12.55$ $[5.13,21.63]$	$12.01$ $[4.84,20.16]$
Real-query calibration	$52.00$ $[40.00,66.00]$	$52.00$ $[40.00,66.00]$	$52.00$ $[40.00,66.00]$
Transport $-$ target-only	$-0.56$ $[-1.98,0.42]$	$-0.14$ $[-0.70,0.30]$	$0.05$ $[-0.64,0.57]$
Transport $-$ full FT	$-3.66$ $[-8.57,0.18]$	$-10.89^{\ast}$ $[-19.82,-4.14]$	$-10.34^{\ast}$ $[-17.88,-3.84]$

Appendix

Table 48: Matched Office-Home CLIP-65 identity (%) and same-query real calibration. Effects are percentage points with 95% class intervals; positive favours transport. The identical real-query set is used at every target budget.

Comparison	Data and sampling	Evaluation and uncertainty
GMM factorial	42 $(\alpha,\eta)$ cells; five seeds; 64 held-out tasks per seed	SW/MMD paired by seed and task; hierarchical intervals; abundant-target encoding reference uses 512 target points.
CIFAR main zero-target	20 validation classes $\times$ three corruptions; 64 source images; 96 generated/128 query images	Pixel SW: 256 projections; energy MMD: 64/64 samples. Frozen semantic instrument from the original package. Class-cluster intervals.
CIFAR original adaptation comparison	20 test classes $\times$ three corruptions at each $K_{T}=1,5,20$ ; 96 generated/128 query images	Pixel SW and energy MMD as above. Budget-specific full-FT settings; class-cluster intervals.
CIFAR conditioning, transport and source-budget ablations	Separately matched validation comparisons; nested source prefixes for $M_{S}=16,\ldots,300$	Each comparison uses its own trained model and matched class–corruption episodes. Paired class-cluster intervals; one training seed per arm.
CIFAR independent direct- $z_{T}$ diagnostic	Its separate 200-support/50-query repartition	Its stated DINO identity instrument and real-query calibration; class intervals. The coordinate-retrieval probe instead resamples supports conditional on its class set.
Office-Home main routes	10 held-out classes; eight repeats; 96 generated images per repeat; 20 fixed real queries per class	DINO feature-space distances and 65-way CLIP identity are distinct metrics. Repeats are averaged within class; class intervals.

Appendix

Table 49: Protocol matrix. Counts refer to actual metric evaluation unless marked as pool capacities. Independent comparisons retain their own statistical units. The new supplementary experiments are described in Appendix F .

Diffusion Model Adaptation

See all 38 papers

Jul 23, 2026

Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning

Fine-tuning large diffusion models for new domains or styles involves a trade-off: improving target-specific generation often degrades the pretrained model's broad generative capability. Existing full and parameter-efficient fine-tuning methods typically handle this trade-off only implicitly. In this work, we propose a novel source-prior-driven selective adaptation method to efficiently fine-tune diffusion models, achieving a favorable trade-off. Our method relies on two key observations: (1) the loss of general generative capability is highly inconsistent across pretrained parameters, and (2) parameters that have a relatively small impact on the model's general generative capability remain structurally inconsistent across layers and parameter types. Motivated by these observations, we first learn a static mask to explicitly identify parameters better suited for downstream adaptation, and then construct structured update strategies for the selected subset. Experiments show that our method achieves a better adaptation-retention trade-off than existing strong baselines.

Yi Xiong, Yuan-Yuan Cheng, Xiao-Ming Fu

University of Science and Technology of China, China

Fine-Tuning Diffusion Model Fine-Tuning Selective Fine-Tuning Parameter-Efficient Fine-Tuning Catastrophic Forgetting Diffusion Model Adaptation

Ask PDF

Sep 27, 2026

Revisiting Diffusion Fine-Tuning for Unsupervised Domain Adaptation

Diffusion-based unsupervised domain adaptation (UDA) improves cross-domain transfer by generating target-specific synthetic data for downstream adaptation. Existing methods are largely designed for single-target adaptation: when a model trained on one labeled source domain must be adapted to multiple unlabeled target domains, they typically require separate diffusion fine-tuning for each source--target pair, causing training, storage, and deployment costs to grow with the number of targets. In this paper, we study multi-target data generation for diffusion-based UDA, where a single source-guided diffusion fine-tuning process is reused to generate target-specific synthetic data for multiple target domains. We propose MUSE (Multi-target UDA-oriented Synthesis with Efficient diffusion fine-tuning), a decoupled adaptation framework that separates source-supervised semantic adaptation from target-specific style adaptation. MUSE uses a shared semantic branch updated by labeled source data and target-private style branches specialized to individual target domains, enabling target-specific generation while avoiding repeated source-guided fine-tuning for each target. Experiments on standard UDA benchmarks show that MUSE achieves a stronger accuracy--efficiency trade-off than repeated per-target diffusion adaptation, reducing diffusion fine-tuning cost while improving average target-domain accuracy. The project page is available at https://xuanqi99.github.io/MUSE/.

Xuan Qi, Yi Wei, Daniele Berardini +2

AI for Good (AIGO), Istituto Italiano di Tecnologia, Genoa, Italy · DITEN, University of Genoa, Genoa, Italy · National Key Laboratory of Novel Software Technology, Nanjing University, China +3

Unsupervised Domain Adaptation Domain Adaptation Diffusion Model Fine-Tuning

Ask PDF

Aug 20, 2026

Continuous Adversarial MeanFlow Transfer

Training fast generators on new domains with limited data remains challenging for two reasons. First, adapting a pretrained diffusion or flow model to a new domain leaves its costly multi-step sampling unaddressed, and existing acceleration methods are tied to the source parameterization--

ε

x

v

, or

u

--leaving heterogeneous pretrained models with no common acceleration target. Second, while adversarial refinement is proven effective for few-step quality, it is formulated only for instantaneous-velocity flows, not for the finite-interval average velocities that MeanFlow (MF) models predict. We address both problems. We propose MeanFlow-Transfer, which maps heterogeneous source outputs into a shared velocity representation, uses it to initialize an MF generator from the source weights, and optimizes an MF objective on the target domain. This unifies adaptation and acceleration in a single training loop across a broad range of pretrained models. We then introduce Continuous Adversarial MeanFlow, a post-training stage that extends continuous adversarial flow models from instantaneous velocities to MF's finite-interval average velocities. CAMF contrasts changes in a learned potential between real and predicted interval endpoints, recovering fine detail that MF regression averages away, and reduces to the instantaneous criterion in the vanishing-interval limit. Adapting four ImageNet-based source models--DiT (

ε

), SiT (

v

), JiT (

x

), iMF (

u

)--to five target domains, MF-T with CAMF matches or exceeds the fine-tuned teacher in FID and FDD at up to

125\times

fewer Neural Function Evaluations (NFEs), while CAMF improves MF-T's few-step FID by

29\%

on average.

Yara Bahram, Zahra Dehghani, Mélodie Desbos +3

LIVIA · ILLS · ÉTS Montreal +2

Diffusion Model Adaptation Flow Matching Diffusion Model Inference Acceleration Domain Adaptation

Ask PDF

Abstract

Diffusion Model Adaptation

Explore similar work

Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning

Revisiting Diffusion Fine-Tuning for Unsupervised Domain Adaptation

Continuous Adversarial MeanFlow Transfer