CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer

Wenbo Nie; Zixiang Li; Renshuai Tao; Bin Wu; Yunchao Wei; Yao Zhao

cs.CVFeb 16, 2026

CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer

Authors: Wenbo Nie, Zixiang Li, Renshuai Tao, Bin Wu, Yunchao Wei, Yao Zhao

Organizations: Institute of Information Science, Beijing Jiaotong University · Visual Intelligence + X International Joint Laboratory of the Ministry of Education

Abstract

Transferring visual style between images while preserving semantic correspondence between similar objects remains a central challenge in computer vision. While existing methods have made great strides, most of them operate at global level but overlook region-wise and even pixel-wise semantic correspondence. To address this, we propose CoCoDiff, a novel training-free and low-cost style transfer framework that leverages pretrained latent diffusion models to achieve fine-grained, semantically consistent stylization. We identify that correspondence cues within generative diffusion models are under-explored and that content consistency across semantically matched regions is often neglected. CoCoDiff introduces a pixel-wise semantic correspondence module that mines intermediate diffusion features to construct a dense alignment map between content and style images. Furthermore, a cycle-consistency module then enforces structural and perceptual alignment across iterations, yielding object and region level stylization that preserves geometry and detail. Despite requiring no additional training or supervision, CoCoDiff delivers state-of-the-art visual quality and strong quantitative results, outperforming methods that rely on extra training or annotations.

Figures & tables

Figure 1: Comparison of different style transfer methods. We compare our proposed CoCoDiff with three other representative methods with zoomed-in details.

Figure 2: Framework Overview of our proposed Correspondence-Consistent Diffusion (CoCoDiff).

Figure 3: Qualitative comparison. We compare CoCoDiff (Ours) with seven representative methods, selected from diffusion-based, patch-based, CNN-based, transformer-based, and other approaches, to provide a comprehensive evaluation.

Figure 4: (A) Qualitative comparison with additional zoomed-in details . We compare our method with StyleID and StyTR 2 as baseline approaches, highlighting the differences through zoomed-in details. (B) The illustration of cycle-based image style transfer. (a) Direct feature matching between the style image and the content image often results in low matching accuracy and feature correspondence failure. (b) By first transforming the style image to adopt the content image’s style before performing feature matching, the matching accuracy is significantly improved. (c) Direct correspondence result. (d) Indirect correspondence result.

Metric	Ours	StyleID	AesPA	StyTR 2	InST	Instant+	AdaIN	SCSA	FreeStyle	SMS
FID $\downarrow$	18.432	21.010	19.645	18.886	21.541	20.982	18.672	20.835	23.654	31.266
LPIPS $\downarrow$	0.549	0.565	0.556	0.587	0.785	0.584	0.612	0.562	0.689	0.821
ArtFID $\downarrow$	30.100	34.446	32.124	31.559	40.235	34.820	31.711	34.106	41.640	58.756
CFSD $\downarrow$	0.609	0.619	0.632	0.687	0.881	0.710	0.642	0.612	0.660	0.704

Table 1: Quantitative evaluation results. We compare every methods across multiple metrics. Columns

2^{nd}

9^{th}

are reference-guided methods while columns

10^{th}

11^{th}

are prompt-based methods.

Table 6

Figure 5: Quantitative comparison of the cycle module. (a) Balance between LPIPS and FID metrics across iterations. (b) CFSD variations across iterations.

Figure 6: Visual results of the iteration process. The group with the best quality is highlighted by a red line.

Table 4: Ablation study analyzing the impact of various design choices on model performance.

Method	Style	Content	Avg
StyleID Chung et al. (2024)	0.212	0.135	0.174
AesPA Hong et al. (2023)	0.033	0.134	0.084
StyTR 2 Deng et al. (2022)	0.132	0.036	0.084
InST Zhang et al. (2023b)	0.004	0.025	0.015
Instant+ Wang et al. (2024a)	0.002	0.174	0.088
AdaIN Huang and Belongie (2017)	0.038	0.013	0.026

Table 5: User study results in consistency and quality.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Additional classic visual outcomes in diverse artistic styles.

Figure 8: Visualization results of our proposed methods.

Figure 9: Comparison of style transfer results. Visual results of our CoCoDiff method compared against four baseline approaches across diverse artistic styles: Ghibli Style, Pixel Art, Chinese Ink, Kid Illustration, and Vintage Style.

Metric	Ours	CNNMRF	Style-Swap	DIA	Avatar-Net	SCSA+StyleID
FID (↓)	18.432	27.872	35.642	31.933	22.356	20.835
LPIPS (↓)	0.549	0.672	0.793	0.661	0.641	0.562
CFSD (↓)	0.609	0.844	0.761	0.649	0.753	0.612

Appendix

Table 6: Comparison of metrics across methods.

Figure 10: Additional comparisons with patch-based methods.

Figure 11: Fine-grained style transfer visual results: Flower.

Figure 12: Fine-grained style transfer visual results: Cow.

Figure 13: Fine-grained style transfer visual results: House.

Cycle Consistency

See all 13 papers

RegionRoute: Regional Style Transfer with Diffusion Model
Feb 22, 2026Bowen Chen, Jake Zuena, Alan C. Bovik +1Image Style Transfer Diffusion-Based Image Editing
The University of Texas at Austin · Dolby Laboratories, Inc.
StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References
Mar 11, 2026Boyu He, Yunfan Ye, Chang Liu +3Image Style Transfer Diffusion-Based Image Editing
College of Computer Science and Technology, National University of Defense Technology · School of Design, Hunan University
SafeStyle: Calibrated Style Residual Injection for Controllable Style-Leakage Trade-off in Diffusion Stylization
Sep 18, 2026Zhangping Yang, Min Li, Song Yan +4Image Style Transfer Diffusion-Based Image Editing
Xi’an High-tech Research Institute, Shaanxi, China · University of Science and Technology of China, Anhui, China · Xi’an University of Architecture and Technology, Shaanxi, China

Feb 22, 2026J/K move · Enter open · S save

RegionRoute: Regional Style Transfer with Diffusion Model

Bowen Chen, Jake Zuena, Alan C. Bovik, Divya Kothandaraman

The University of Texas at Austin · Dolby Laboratories, Inc.

Image Style Transfer Diffusion-Based Image Editing Attention Control in Diffusion Models Diffusion Model Fine-Tuning

Open paper AskPDF arXiv

Precise spatial control in diffusion-based style transfer remains challenging. This challenge arises because diffusion models treat style as a global feature and lack explicit spatial grounding of style representations, making it difficult to restrict style application to specific objects or regions. To our knowledge, existing diffusion models are unable to perform true localized style transfer, typically relying on handcrafted masks or multi-stage post-processing that introduce boundary artifacts and limit generalization. To address this, we propose an attention-supervised diffusion framework that explicitly teaches the model where to apply a given style by aligning the attention scores of style tokens with object masks during training. Two complementary objectives, a Focus loss based on KL divergence and a Cover loss using binary cross-entropy, jointly encourage accurate localization and dense coverage. A modular LoRA-MoE design further enables efficient and scalable multi-style adaptation. To evaluate localized stylization, we introduce the Regional Style Editing Score, which measures Regional Style Matching through CLIP-based similarity within the target region and Identity Preservation via masked LPIPS and pixel-level consistency on unedited areas. Experiments show that our method achieves mask-free, single-object style transfer at inference, producing regionally accurate and visually coherent results that outperform existing diffusion-based editing approaches.

Abstract

Cycle Consistency

Explore similar work

RegionRoute: Regional Style Transfer with Diffusion Model

StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References

SafeStyle: Calibrated Style Residual Injection for Controllable Style-Leakage Trade-off in Diffusion Stylization