Mixture of Style Experts for Diverse Image Stylization

Shihao Zhu; Ziheng Ouyang; Yijia Kang; Qilong Wang; Mi Zhou; Bo Li; Ming-Ming Cheng; Qibin Hou

cs.CVMar 17, 2026

Mixture of Style Experts for Diverse Image Stylization

Authors: Shihao Zhu, Ziheng Ouyang, Yijia Kang, Qilong Wang, Mi Zhou, Bo Li, Ming-Ming Cheng, Qibin Hou

Organizations: VCIP, CS, Nankai University · Tianjin University · vivo BlueImage Lab · NKIARI, Shenzhen Futian

Abstract

Diffusion-based stylization has advanced significantly, yet existing methods are limited to color-driven transformations, neglecting complex semantics and material details. We introduce StyleExpert, a semantic-aware framework based on the Mixture of Experts (MoE). Our framework employs a unified style encoder, trained on our large-scale dataset of content-style-stylized triplets, to embed diverse styles into a consistent latent space. This embedding is then used to condition a similarity-aware gating mechanism, which dynamically routes styles to specialized experts within the MoE architecture. Leveraging this MoE architecture, our method adeptly handles diverse styles spanning multiple semantic levels, from shallow textures to deep semantics. Extensive experiments show that StyleExpert outperforms existing approaches in preserving semantics and material details, while generalizing to unseen styles. Our code and collected images are available at the project page: https://hh-lg.github.io/StyleExpert-Page/.

Figures & tables

Figure 2: Overview of the proposed StyleExpert. Our method comprises two training stages. In the first stage (Sec. 3.2 ), a style encoder is trained with the InfoNCE loss to learn discriminative style representations, thereby accelerating convergence. In the second stage (Sec. 3.3 ), the pre-trained encoder provides style priors to guide the router network in training MoE LoRA adapters, enabling each layer to dynamically select the most suitable experts for diverse styles.

Figure 3: Overview of StyleExpert-500K. (a) illustrates the hierarchical distribution of all 209 styles in StyleExpert-500K, where bar heights indicate the number of styles per category. (b) presents examples of nine different stylizations for the same content from our StyleExpert-500K dataset. (c) compares the focus on semantic stylization between StyleExpert-500K and OmniStyle-150K.

Figure 4: Overview of our dataset curation pipeline. We first manually collect content images and style LoRAs from the web. For each content image, we use Qwen [ 2 ] to generate a clean caption that excludes color and style information. The content image, its caption, and the corresponding style LoRA are then fed into the model to generate stylized images. A vision-language model (VLM) filters out those with incorrect or failed stylization. Finally, we compute CLIP [ 36 ] similarity to select the most suitable style reference for each target, forming the final triplet dataset.

Figure 5: Qualitative comparison of our method, StyleExpert, with other SOTA style transfer methods on unseen styles.

	CLIP	DINO	CSD	Aesthetic	Qwen Semantic	DreamSim
Method	( $\uparrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\downarrow$ )
CSGO	63.41	65.50	61.07	6.28	28.93	42.39
DreamO	64.14	62.68	47.91	6.20	19.29	44.95
OmniGen2	67.07	63.06	55.61	6.13	23.69	41.55
OmniStyle	65.39	7 2.27	59.65	6.07	40.00	41.83
Qwen-Image-Edit	67.47	55.86	56.74	6.20	42.74	3 4.47

Table 1: Quantitative comparison of our method against other stylization methods, as well as LoRA-only and MoE-only fine-tuning baselines. Best results are marked in bold and second-best results are underlined . All metrics except for the Aesthetic score are presented as percentages(%).

	Computation (G)	Trainable Params (M)
Base Model	10.92	-
+ LoRA	+0.67	751.48
+ MoE Experts	+0.12	818.71

Table 2: Computation and Parameter Comparison

Figure 6: Impact of the style encoder on MoE Training.

Figure 7: Qualitative comparison of our method against LoRA-only and MoE-only fine-tuning baselines.

Figure 8: t-SNE visualization of style embeddings from our style encoder on the test set.

Model Stage	Similar Styles	Dissimilar Styles
Early Stage	34.01%	9.77%
Mid Stage	34.36%	10.32%
Late Stage	31.15%	11.70%
Overall Avg.	33.18%	10.60%

Table 3: Analysis of MoE expert routing overlap (IoU) between similar styles and dissimilar styles generation.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Illustration of our prompt filtering process. Initial VL-generated captions often retain descriptors ( e.g . , colors, atmosphere) that conflict with styles defined by their own intrinsic properties. This is detrimental to styles with unique color palettes ( e.g . , blue dot, Digital Yellow, Van Gogh, Elegant Line) or distinct atmospheres ( e.g . , Pop Art, 3D Texture, Cinematic, Contra). Our rewriting step removes these conflicting descriptors to produce clean prompts, ensuring consistent stylization.

	Color-Semantic	CLIP	DINO	CSD	Aesthetic	DreamSim
Dataset	Balance	( $\uparrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\uparrow$ )	( $\downarrow$ )
OmniStyle-150K	✗	67.68	67.71	38.47	6.08	59.49
StyleExpert-500K	✓	68.41	66.75	74.49	6.56	36.19

Appendix

Table 4: Quantitative comparison between OmniStyle-150K and our StyleExpert-500K.

	CSGO	DreamO	OmniGen2	OmniStyle	QwenEdit	USO	StyleExpert
Top1	3.0	1.1	8.5	4.8	3.6	4.5	74.5

Appendix

Table 5: User study results. Values indicate the Top-1 preference rate (%) across all methods.

Figure 10: Comparison of data samples from our StyleExpert-500K dataset and the OmniStyle-150K dataset.

Figure 11: Qualitative comparison of our method against competing approaches under various content and unseen style inputs.

Figure 12: Qualitative comparison of our method against competing approaches under various content and unseen style inputs.

Figure 13: Additional visualization results generated by our method across a diverse range of styles and content subjects.

Figure 14: Additional visualization results generated by our method across a diverse range of styles and content subjects.

Figure 15: Additional visualization results generated by our method across a diverse range of styles and content subjects.

Figure 16: Additional visualization results generated by our method across a diverse range of styles and content subjects.

Image Style Transfer

Image style transfer changes an image’s visual appearance to match an artistic, photographic, or other reference while preserving its content.

See all 61 papers

StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References
Mar 11, 2026Boyu He, Yunfan Ye, Chang Liu +3Image Style Transfer Diffusion-Based Image Editing
College of Computer Science and Technology, National University of Defense Technology · School of Design, Hunan University
CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer
Feb 16, 2026Wenbo Nie, Zixiang Li, Renshuai Tao +3Cycle Consistency Image Style Transfer
Institute of Information Science, Beijing Jiaotong University · Visual Intelligence + X International Joint Laboratory of the Ministry of Education
Scale-Separated Conditioning for Style-Encoder-Free Diffusion Stylization
Aug 20, 2026Jingtao Zhang, Haorui Gao, Youqing Liang +1Image Style Transfer Conditional Diffusion Models
College of Computing, Georgia Institute of Technology, Atlanta, GA, USA · Courant Institute of Mathematical Sciences, New York University, New York, NY, USA · Department of Computer Science, Brown University, Providence, RI, USA

Mar 11, 2026J/K move · Enter open · S save

StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References

Boyu He, Yunfan Ye, Chang Liu, Weishang Wu+2

College of Computer Science and Technology, National University of Defense Technology · School of Design, Hunan University

Image Style Transfer Diffusion-Based Image Editing

Open paper AskPDF arXiv

Despite the advancements in diffusion-based image style transfer, existing methods are commonly limited by 1) semantic gap: the style reference could miss proper content semantics, causing uncontrollable stylization; 2) reliance on extra constraints (e.g., semantic masks) restricting applicability; 3) rigid feature associations lacking adaptive global-local alignment, failing to balance fine-grained stylization and global content preservation. These limitations, particularly the inability to flexibly leverage style inputs, fundamentally restrict style transfer in terms of personalization, accuracy, and adaptability. To address these, we propose StyleGallery, a training-free and semantic-aware framework that supports arbitrary reference images as input and enables effective personalized customization. It comprises three core stages: semantic region segmentation (adaptive clustering on latent diffusion features to divide regions without extra inputs); clustered region matching (block filtering on extracted features for precise alignment); and style transfer optimization (energy function-guided diffusion sampling with regional style loss to optimize stylization). Experiments on our introduced benchmark demonstrate that StyleGallery outperforms state-of-the-art methods in content structure preservation, regional stylization, interpretability, and personalized customization, particularly when leveraging multiple style references.

Abstract

Image Style Transfer

Explore similar work

StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References

CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer

Scale-Separated Conditioning for Style-Encoder-Free Diffusion Stylization