StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References

Boyu He; Yunfan Ye; Chang Liu; Weishang Wu; Fang Liu; Zhiping Cai

cs.CVMar 11, 2026

StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References

Authors: Boyu He, Yunfan Ye, Chang Liu, Weishang Wu, Fang Liu, Zhiping Cai

Organizations: College of Computer Science and Technology, National University of Defense Technology · School of Design, Hunan University

Abstract

Despite the advancements in diffusion-based image style transfer, existing methods are commonly limited by 1) semantic gap: the style reference could miss proper content semantics, causing uncontrollable stylization; 2) reliance on extra constraints (e.g., semantic masks) restricting applicability; 3) rigid feature associations lacking adaptive global-local alignment, failing to balance fine-grained stylization and global content preservation. These limitations, particularly the inability to flexibly leverage style inputs, fundamentally restrict style transfer in terms of personalization, accuracy, and adaptability. To address these, we propose StyleGallery, a training-free and semantic-aware framework that supports arbitrary reference images as input and enables effective personalized customization. It comprises three core stages: semantic region segmentation (adaptive clustering on latent diffusion features to divide regions without extra inputs); clustered region matching (block filtering on extracted features for precise alignment); and style transfer optimization (energy function-guided diffusion sampling with regional style loss to optimize stylization). Experiments on our introduced benchmark demonstrate that StyleGallery outperforms state-of-the-art methods in content structure preservation, regional stylization, interpretability, and personalized customization, particularly when leveraging multiple style references.

Figures & tables

Figure 2: Overall framework. Our pipeline comprises three stages: (a) In stage 1, the content image is diffused for T steps to extract UNet features ${F_{0},\dots,F_{T}}$ , which are weighted into $F_{mix}$ , then clustered via PCA and K-means to optimize the mask. (b) In stage 2, $F_{mix}$ and the semantic mask identify cluster features, which are aggregated via self-attention for statistical similarity. Meanwhile, DINOv2 [ 32 ] splits $x_{0}$ into blocks; tokens are filtered for semantic similarity, while cluster positions yield positional similarity. (c) Stage 3 optimizes the generation through $N$ latent sampling steps. UNet attention maps are extracted, sparsified using the semantic mask, and recombined with style features ( $K_{s}$ , $V_{s}$ ). L1 distances are then computed between the actual and combined feature maps, as well as the $Q$ and $Q_{c}$ . These losses guide the final image generation.

Figure 3: Cluster optimization. We compute pairwise semantic distances among clusters, merge those below a threshold (set to 0.85), then split–recombine clusters guided by the input’s depth features, then traverse each pixel, eliminate isolated points.

Figure 4: Sparse attention based on semantic matches. We mask the attention feature map, preserving weights for relevant semantics and zeroing out irrelevant regions. Content-sparse attention matches corresponding style features, yielding optimal stylization. Semantically identical regions are color-coded; connections use matching colors.

Figure 5: Qualitative comparisons with recent state-of-the-art image style transfer methods. Our StyleGallery achieves semantic-level style transfer for fine-grained content (e.g., hair and clothing in the first row; sea and sailboat in the second row) while preserving content consistency, and others either fail to transfer styles accurately or compromise content structure. Regarding multiple references, our method can accurately and adaptively recognize, match, and transfer regional styles (the third row) without introducing incorrect semantic content seen in others (e.g., unexpected semantics in the sky of other methods in the fourth row). Zoom-in is recommended for better comparisons.

Metric	CAST [ 49 ]	CCPL [ 44 ]	AdaAttn [ 29 ]	StyTR-2 [ 6 ]	CSGO [ 45 ]	StyleShot [ 8 ]	StyleID [ 4 ]	AD [ 53 ]	AttnST [ 16 ]	Ours
Style ↑	0.4959	0.5163	0.5094	0.5219	0.5224	0.5198	0.4972	0.5249	0.5032	0.5337
Gram Loss ↓	15.778	17.8606	17.357	16.719	19.937	19.013	14.261	13.862	16.937	13.519
FID [ 15 ] ↓	17.255	18.141	18.498	17.623	19.829	20.638	18.987	17.677	19.233	16.889
LPIPS [ 48 ] ↓	0.4934	0.4549	0.5299	0.3856	0.5005	0.6615	0.4496	0.4032	0.4532	0.3716
ArtFID [ 43 ] ↓	27.262	27.843	29.831	25.804	31.254	35.952	28.973	26.207	29.402	24.536

Table 1: Quantitative comparison with image style transfer methods. The best results are shown in bold , and the second-best results are indicated with an underline .

Figure 6: Zoomed details of our method, AD [ 53 ] and StyleID [ 4 ] . Black boxes highlight enlarged views of the results.AD [ 53 ] under-stylizes the image, with the background remaining unchanged, while StyleID [ 4 ] introduces incorrect patterns. This underscores the need for our semantic-aware regional style transfer.

Figure 7: Comparisons with mask-based method SCSA [ 38 ] .

Figure 8: Ablation study on loss functions. The global content loss preserves semantic consistency while the region style loss enables effective transfer, with their combination yielding faithful stylization; additionally, confining style transfer to semantically matched region pairs further enhances quality and mitigates semantic leakage in the generated images.

Configuration	LPIPS [ 48 ] ↓	FID [ 15 ] ↓	Style ↑
Ours ( $\lambda_{c}=0.26$ , default)	0.3716	16.89	0.5337
Ours ( $\lambda_{c}=0.29$ , above)	0.3689	27.26	0.4172
Ours ( $\lambda_{c}=0.22$ , below)	0.4354	21.84	0.4562
w/o RSL	0.5195	23.56	0.4387
w/o GCL	0.6822	30.83	0.4150

Table 2: Ablation study on our proposed components.

Figure 9: Visual impact of the content loss weight $\lambda_{c}$ . As the global content loss rises from left to right, generated results better preserve the content image’s structure while keeping stylization. In automatic mode, we set 0.26 as the default; in custom mode, this weight can be selected by the user.

Figure 10: Compatibility analysis. We integrated with accelerated models reduces optimization steps from 150 to 28. The reduced inference times (highlighted in green) demonstrate a significant speedup while maintaining high-fidelity stylization quality.

Figure 11: Results of robustness analysis.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Overall composition of the style dataset.

Figure 2: Category distribution in style dataset.

Figure 3: Ablations on cluster merge threshold.

Threshold	0.55	0.65	0.75	0.85	0.95
Accuracy $\approx$ ( $\%$ ) ↑	52	65	77	93	90

Appendix

Table 1: Ablation study: Effect of merge threshold on semantic classification accuracy. The best performance (93%) is achieved at threshold 0.85.

Metric	$\lambda_{1}$ =0.05	$\lambda_{1}$ =0.25	$\lambda_{1}$ =0.85	only $\lambda_{2}$	$\lambda_{2}+\lambda_{1}$	$\lambda_{2}+\lambda_{1}+\lambda_{3}$
LPIPS [ 48 ] ↓	0.2345	0.2317	0.2323	0.2413	0.2321	0.2317
Accuracy $\approx$ ( $\%$ ) ↑	86	89	87	87	89	89

Appendix

Table 2: Ablation study on similarity weight. It shows that values of

\lambda_{1}

(low or high) may yield erroneous semantic correspondences, and indicate that relying solely on semantic features extracted by DINOv2 [ 32 ] can lead to mismatches, whereas incorporating statistical features further improves matching accuracy.

Figure 4: Ablations on optimization steps.

Metric	StyleID	AD	Ours
Visual	3.76	3.77	4.43
Region	3.44	3.75	4.41
Content	4.35	3.73	4.46

Appendix

Table 3: User preference scores across three aspects.

	Distribution (%)
CS	76
Non-CS	24

Appendix

Table 4: User distribution.

Metric	StyleID	AD	Ours
Visual	3.75	3.87	4.50
Region	3.36	3.85	4.60
Content	4.40	3.81	4.54

Appendix

Table 5: Left: CS background users; Right: non-CS background users.

Figure 6: User personalization gradio interface.

Figure 7: Generation results under different backbones.

Figure 11: More personalized style transfer results.

Figure 12: More qualitative comparison results with large models.

Image Style Transfer

Image style transfer changes an image’s visual appearance to match an artistic, photographic, or other reference while preserving its content.

See all 60 papers

CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer
Feb 16, 2026Wenbo Nie, Zixiang Li, Renshuai Tao +3Cycle Consistency Image Style Transfer
Institute of Information Science, Beijing Jiaotong University · Visual Intelligence + X International Joint Laboratory of the Ministry of Education
SafeStyle: Calibrated Style Residual Injection for Controllable Style-Leakage Trade-off in Diffusion Stylization
Sep 18, 2026Zhangping Yang, Min Li, Song Yan +4Image Style Transfer Diffusion-Based Image Editing
Xi’an High-tech Research Institute, Shaanxi, China · University of Science and Technology of China, Anhui, China · Xi’an University of Architecture and Technology, Shaanxi, China
TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer
Oct 4, 2026Duc Khoan Le, Kim Ngoc Tran, Minh Nhat Le +5Image Style Transfer Diffusion Model Guidance
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam

Feb 16, 2026J/K move · Enter open · S save

CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer

Wenbo Nie, Zixiang Li, Renshuai Tao, Bin Wu+2

Institute of Information Science, Beijing Jiaotong University · Visual Intelligence + X International Joint Laboratory of the Ministry of Education

Cycle Consistency Image Style Transfer Diffusion-Based Image Editing Semantic Correspondence

Open paper AskPDF arXiv

Transferring visual style between images while preserving semantic correspondence between similar objects remains a central challenge in computer vision. While existing methods have made great strides, most of them operate at global level but overlook region-wise and even pixel-wise semantic correspondence. To address this, we propose CoCoDiff, a novel training-free and low-cost style transfer framework that leverages pretrained latent diffusion models to achieve fine-grained, semantically consistent stylization. We identify that correspondence cues within generative diffusion models are under-explored and that content consistency across semantically matched regions is often neglected. CoCoDiff introduces a pixel-wise semantic correspondence module that mines intermediate diffusion features to construct a dense alignment map between content and style images. Furthermore, a cycle-consistency module then enforces structural and perceptual alignment across iterations, yielding object and region level stylization that preserves geometry and detail. Despite requiring no additional training or supervision, CoCoDiff delivers state-of-the-art visual quality and strong quantitative results, outperforming methods that rely on extra training or annotations.

Abstract

Image Style Transfer

Explore similar work

CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer

SafeStyle: Calibrated Style Residual Injection for Controllable Style-Leakage Trade-off in Diffusion Stylization

TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer