MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer

Dongkyung Kang; Jaeyeon Hwang; Junseo Park; Minji Kang; Yeryeong Lee; Beomseok Ko; Hanyoung Roh; Jeongmin Shin

cs.CVApr 14, 2026

MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer

Authors: Dongkyung Kang, Jaeyeon Hwang, Junseo Park, Minji Kang, Yeryeong Lee, Beomseok Ko, Hanyoung Roh, Jeongmin Shin,

Organizations: Department of Computer Science & Artificial Intelligence, Dongguk University

Abstract

Style transfer applies the appearance of a reference image to a content image while preserving its spatial structure. Recent diffusion-based methods achieve strong stylization but typically assume a single global style. We instead consider regional-multi style transfer, which assigns multiple references to user-specified regions of a content image. Extending them to this setting reveals two coupled shared-attention issues: ambiguous mass allocation among content and style partitions and degraded selectivity as more styles are jointly normalized, while style aggregation at the attention output further suppresses fine details. We propose MAST (Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer), a unified attention-control framework for frozen diffusion models. Logit-level Attention Mass Allocation enforces mask-derived partition masses, Sharpness-aware Temperature Scaling adaptively restores selectivity, and Discrepancy-aware Detail Injection recovers high-frequency content. MAST jointly processes all style--mask pairs in a single denoising pass without training, optimization, or post-hoc composition. Across two to five styles, MAST achieves the best average ArtFID, FID, and R-FID among all baselines, demonstrating regional style fidelity, content preservation, and scalability.

Figures & tables

Panel of Figure 2 — Figure 2: The two shared-attention failure modes and their corrections. (A) Under shared attention, the attention mass assigned to the upper style reference spreads beyond the target region; LAMA focuses it on the mask. (B) Rank-wise attention becomes flatter as the number of style references $N$ increases, whereas STS preserves selectivity (left); the corresponding sharpness maps (right) further illustrate this improvement.

Figure 3: Overview of the MAST framework. (Left) The stylization pipeline employs DDIM inversion and AdaIN-based initialization, with style injected through attention modulation during denoising. (Right) The attention control framework comprises two core attention-control modules - LAMA (mass allocation) and STS (attention sharpening) - and one complementary detail-preservation module (DDI). The bar charts illustrate attention allocation progressively reshaped by LAMA and sharpened by STS.

Method	ArtFID $\downarrow$					FID $\downarrow$					LPIPS $\downarrow$					R-FID $\downarrow$
Method	2	3	4	5	Avg	2	3	4	5	Avg	2	3	4	5	Avg	2	3	4	5	Avg
(G-S)
$Z^{*}$	17.215	17.009	17.253	16.926	17.101	10.865	10.745	10.876	10.664	10.787	0.451	0.448	0.453	0.451	0.451	13.153	16.693	21.566	23.062	18.618
StyleID	16.635	16.819	16.995	16.927	16.844	10.323	10.451	10.643	10.383	10.450	0.469	0.469	0.460	0.487	0.471	13.685	17.956	25.562	26.523	20.931
CoCoDiff	16.870	17.217	17.345	17.308	17.185	10.400	10.633	10.555	10.455	10.511	0.480	0.480	0.501	0.511	0.493	13.680	18.219	26.322	27.152	21.343
(R-S)

Table 1: Quantitative comparison across two to five reference styles. Methods are grouped into Global-Single ( G-S ), Regional-Single ( R-S ), and Regional-Multi ( R-M ) style transfer. For methods that composite single-style outputs, the best of hard masking and three soft-blending schemes is reported, selected per setting by ArtFID. Avg denotes the arithmetic mean over the four style counts.

Figure 4: Qualitative comparison across style settings, illustrating baseline failure modes and MAST ’s coherent multi-style results. Rows 1–4 correspond to $N=2,3,4,5$ , respectively. Columns 4–6, 7–8, and 9–10 show the G-S , R-S , and R-M baselines, respectively. Each G-S and R-S baseline uses its best-performing compositing strategy. See Fig. 1 for the mask notation.

Figure 5: Progressive multi-style transfer under a fixed four-style mask. Styles are sequentially added from left to right.

Method	ArtFID $\downarrow$				FID $\downarrow$				LPIPS $\downarrow$
Method	2	3	4	5	2	3	4	5	2	3	4	5
StyleMixer	16.461	17.025	17.661	17.665	9.646	10.087	10.571	10.595	0.546	0.536	0.526	0.524
StyleGallery	15.033	15.234	15.359	15.321	9.240	9.445	9.554	9.540	0.468	0.458	0.455	0.454
MAST	14.780	14.802	15.014	15.117	8.656	8.700	8.905	8.929	0.531	0.526	0.516	0.522

Table 2: Quantitative comparison with automatic semantic-matching methods.

Figure 6: Comparison with regional-multi baselines under automatic semantic-matching. StyleMixer (top) and StyleGallery (bottom) are shown with their inferred masks (pink and green); MAST uses the same inferred masks for region-wise stylization.

Figure 7: Qualitative ablation of MAST components in the two-style setting.

LAMA	STS	DDI	ArtFID $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	R-FID $\downarrow$
			$30.549$	$24.134$	$0.216$	$33.930$
$\checkmark$			$16.680$	$9.665$	$0.564$	$13.276$
$\checkmark$	$\checkmark$		14.769	8.495	$\underline{0.555}$	10.803
$\checkmark$	$\checkmark$	$\checkmark$	$\underline{14.780}$	$\underline{8.656}$	0.531	$\underline{11.260}$

Table 3: Quantitative ablation of MAST components in the two-style setting; the base setting is excluded from ranking.

Method	Style	Content	Leakage $\downarrow$	TV $\downarrow$	JSD $\downarrow$
Shared Attn.	0.242	0.582	0.175	0.656	0.276
+ LAMA	$\mathbf{0.900}$	$\mathbf{0.100}$	$\mathbf{1.0\times 10^{-6}}$	$\mathbf{9.1\times 10^{-8}}$	$\mathbf{1.6\times 10^{-8}}$
Target	0.900	0.100	0	0	0

Table 4: Partition-allocation ambiguity in the two-style setting over

200

images, resolved by LAMA. Leakage, TV, and JSD reach numerical precision (

<10^{-5}

) under LAMA.

Panel of Figure 8 — Figure 8: Comparison of fixed and adaptive sharpening in the two-style setting.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

$\lambda$	ArtFID $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	R-FID $\downarrow$
0.50	16.690	10.459	0.457	14.147
0.25	15.103	8.960	0.516	11.735
0.20	14.780	8.656	0.531	11.260
0.15	14.645	8.476	0.546	10.828
0.10	14.357	8.186	0.563	10.112

Appendix

Table 5: Effect of the content anchoring parameter

\lambda

on performance.

$\pi^{\star}$	ArtFID $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	R-FID $\downarrow$
0.30	28.920	22.308	0.241	30.472
0.45	27.429	20.601	0.270	28.425
0.60	24.827	17.816	0.319	24.984
0.75	21.035	14.066	0.396	22.470
0.90	14.780	8.656	0.531	11.260
1.00	15.235	8.890	0.540	11.344

Appendix

Table 6: Quantitative evaluation in the two-style setting across different values of

\pi^{\star}

Figure 9: Empirical mapping from the sharpness gap $\Delta$ to the reference-matching factor $\tau^{\star}$ . Points denote head-level observations, the red curve denotes the fitted quadratic function, and the dashed line marks the lower boundary $\tau=0.8$ of the calibration grid.

Degree	$R^{2}$ $\uparrow$
$1$	$0.927$
$2$	$0.932$
$3$	$0.934$
$4$	$0.935$

Appendix

Table 7: Goodness of fit for polynomial mappings of different degrees.

Figure 10: Entropy-based verification of STS in the four-style setting. Mean attention entropy for content self-attention, the LAMA-adjusted joint attention before STS, and the resulting attention after STS. Values for each head are averaged over spatial queries, controlled layers, and $50$ DDIM timesteps. Lower entropy indicates sharper attention.

$N$	$\mathbb{E}[\widehat{\pi}_{c}^{\mathrm{STS}}]$	$D_{\mathrm{mass}}\downarrow$	$\rho_{\min}\uparrow$
$2$	$0.080$	$0.038$	$0.995$
$3$	$0.078$	$0.035$	$0.998$
$4$	$0.078$	$0.035$	$0.995$
$5$	$0.078$	$0.036$	$0.993$

Appendix

Table 8: Effect of STS on the LAMA-guided allocation. The mean LAMA target for the content partition is

\mathbb{E}[\pi_{c}^{\star}]=0.100

\rho_{\min}

measures agreement with the mask-induced query-wise allocation.

Figure 11: Sensitivity of DDI to the Gaussian frequency scale $r$ . The independently min–max-normalized metrics ( $0$ denotes best) are plotted against the relative squared-gain transmission $T(r)$ on the lower logarithmic axis; the upper axis reports $r$ , and the dashed line marks the default $r=0.3$ .

$r$	ArtFID $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	R-FID $\downarrow$
0.1	15.276	9.126	0.509	12.055
0.3	14.780	8.656	0.531	11.260
0.5	14.707	8.527	0.544	10.984
0.7	14.626	8.451	0.548	10.868
0.9	14.570	8.400	0.550	10.828

Appendix

Table 9: Sensitivity analysis of the normalized Gaussian frequency scale

r

in the two-style setting. Lower values indicate better performance.

Metric	2-style	3-style	4-style	5-style
ArtFID	0.881	0.929	0.952	0.952
FID	0.881	0.881	0.881	0.881
LPIPS	$-0.690$	$-0.762$	$-0.738$	$-0.571$

Appendix

Table 10: Spearman rank correlation between R-FID and complementary metrics across the eight evaluated methods.

Metric	2	3	4	5
M-FID	17.073	27.735	35.915	41.007
ArtFID	14.780	14.802	15.014	15.117
FID	8.656	8.700	8.905	8.929
R-FID	11.260	14.219	18.797	20.883

Appendix

Table 11: Comparison of M-FID, ArtFID, FID, and R-FID on MAST outputs across different numbers of styles.

$\pi^{\star}$	R-FID $\downarrow$
0.10	71.291
0.50	60.264
0.90	18.797

Appendix

Table 12: Controlled style-weakening test in the four-style MAST setting. Only

\pi^{\star}

is varied.

Method	Strategy	$N=2$	$N=3$	$N=4$	$N=5$
$Z^{\ast}$	Hard-mask	17.311	17.009	17.253	16.926
	Alpha	17.215	17.381	17.602	17.530
	Poisson	17.476	17.484	17.324	17.295
	Laplacian-pyramid	17.265	17.449	17.436	17.680
StyleID	Hard-mask	16.866	17.141	17.072	17.009
	Alpha	16.935	17.222	17.371	17.224

Appendix

Table 13: ArtFID comparison of four post-hoc compositing strategies for G-S and R-S baselines. Lower values are better, and bold indicates the strategy selected for each method and number of styles.

Figure 12: User-study interface showing the content image, style references, and method outputs rated using six 7-point Likert criteria.

Figure 13: User study radar charts for eight methods. (a)–(h) show per-method average scores, and (i) overlays all methods. Mean Likert scores are divided by $7$ ; radial axes span $0.30$ – $0.95$ to make differences visible.

Figure 14: Qualitative scalability of MAST across $1$ – $4$ styles, multiple aspect ratios, and $512$ – $1024$ -pixel resolutions ( green : background).

Figure 15: Progressive addition of up to five styles under a fixed five-region spatial partition ( green : background).

Figure 16: Compositional interaction in the two-style setting ( green : background).

Figure 17: Regional style-fidelity comparison with the lowest-ArtFID G-S , R-S , and R-M baselines in the two-style setting ( green : background region). Enlarged views pair each style reference with its corresponding stylized region.

Image Style Transfer

Image style transfer changes an image’s visual appearance to match an artistic, photographic, or other reference while preserving its content.

See all 60 papers

RegionRoute: Regional Style Transfer with Diffusion Model
Feb 22, 2026Bowen Chen, Jake Zuena, Alan C. Bovik +1Image Style Transfer Diffusion-Based Image Editing
The University of Texas at Austin · Dolby Laboratories, Inc.
TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer
Oct 4, 2026Duc Khoan Le, Kim Ngoc Tran, Minh Nhat Le +5Image Style Transfer Diffusion Model Guidance
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
HAM: A Training-Free Style Transfer Approach via Heterogeneous Attention Modulation for Diffusion Models
Mar 25, 2026Yeqi He, Liang Li, Zhiwen Yang +3Image Style Transfer Attention Control in Diffusion Models
Hangzhou Dianzi University · Institute of Computing Technology, Chinese Academy of Sciences · Macao Polytechnic University

Feb 22, 2026J/K move · Enter open · S save

RegionRoute: Regional Style Transfer with Diffusion Model

Bowen Chen, Jake Zuena, Alan C. Bovik, Divya Kothandaraman

The University of Texas at Austin · Dolby Laboratories, Inc.

Image Style Transfer Diffusion-Based Image Editing Attention Control in Diffusion Models Diffusion Model Fine-Tuning

Open paper AskPDF arXiv

Precise spatial control in diffusion-based style transfer remains challenging. This challenge arises because diffusion models treat style as a global feature and lack explicit spatial grounding of style representations, making it difficult to restrict style application to specific objects or regions. To our knowledge, existing diffusion models are unable to perform true localized style transfer, typically relying on handcrafted masks or multi-stage post-processing that introduce boundary artifacts and limit generalization. To address this, we propose an attention-supervised diffusion framework that explicitly teaches the model where to apply a given style by aligning the attention scores of style tokens with object masks during training. Two complementary objectives, a Focus loss based on KL divergence and a Cover loss using binary cross-entropy, jointly encourage accurate localization and dense coverage. A modular LoRA-MoE design further enables efficient and scalable multi-style adaptation. To evaluate localized stylization, we introduce the Regional Style Editing Score, which measures Regional Style Matching through CLIP-based similarity within the target region and Identity Preservation via masked LPIPS and pixel-level consistency on unedited areas. Experiments show that our method achieves mask-free, single-object style transfer at inference, producing regionally accurate and visually coherent results that outperform existing diffusion-based editing approaches.

Abstract

Image Style Transfer

Explore similar work

RegionRoute: Regional Style Transfer with Diffusion Model

TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer

HAM: A Training-Free Style Transfer Approach via Heterogeneous Attention Modulation for Diffusion Models