Style transfer aims to render target content in the style of a reference image, but existing methods often suffer from content leakage, where objects, layouts, or semantics from the style reference appear in the generated output. Although prior data-driven and training-free methods can reduce leakage, they often face a leakage-degradation dilemma: stronger content suppression may weaken style fidelity, while richer style preservation may reintroduce unwanted reference content. We identify this dilemma across the full style-transfer pipeline, including feature separation, feature-space grounding, and diffusion generation. To address these issues, we propose CLeaR, a training-free framework for content-leakage-resistant style transfer. CLeaR first uses Orthogonal Subspace Projection to define content-reduced style targets in each vision foundation model (VFM) feature space. It then performs Ensemble Inversion, which optimizes a shared pixel-space style anchor satisfying style constraints across multiple VFMs. Finally, Energy-Guided Calibration maintains style alignment during diffusion sampling by steering the denoising trajectory toward the ensemble-defined style manifold. We further provide a theoretical analysis showing that the style-anchor estimation error decreases with the number of VFMs. Experiments on StyleBench demonstrate that CLeaR improves style alignment, reduces content leakage, and achieves better LLM-as-Judge evaluation compared with existing methods. The code is available at \href{https://github.com/0606zt/CLeaR}{https://github.com/0606zt/CLeaR}.
Figures & tables
Figure 1: Overview of the CLeaR framework. The three components target the three failure sources in the entire style transfer process: Orthogonal Subspace Projection (feature separation), Ensemble Inversion (feature-space grounding), and Energy-Guided Calibration (diffusion generation).
Figure 2: Visualization of style anchor images extracted by Ensemble Inversion. For each example, the left image is the reference and the right image is the style anchor. The anchor successfully achieves C-S disentanglement, retaining stylistic elements (e.g., color, texture, brushstroke) while suppressing content semantics.
Figure 3: Qualitative comparison results. Existing methods suffer from either content leakage or style degradation , while our method achieves robust stylization without either problem.
Figure 4: Style anchor extraction with varying content descriptions c . For a given reference, different c yield different content images Ic and style anchors Ia∗ . Removing an object from c preserves that object in Ia∗ , showing flexible disentanglement control.
Figure 5: Ablation of VFM compositions. CLIP, DINO, and Inception excel at suppressing content leakage, while VGG and CSD-CLIP better preserve fine-grained style patterns. Increasing the number of models improves both metrics, with the full ensemble of five achieving the best performance.
Figure 6: Effect of guidance stage in Energy-Guided Calibration. We compare applying guidance in the chaotic, semantic, and refinement stages, with the x-axis indicating the guided proportion of each stage. Guidance is ineffective in the chaotic stage, while in the semantic and refinement stages it improves SA with little change in CL. The best efficiency-performance trade-off is achieved by guiding the final 20% of the refinement stage, equivalent to the final T/10 steps.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1: Extended qualitative comparisons, with boxes highlighting content leakage and style degradation . Our method maintains balanced stylization across all examples.
Figure A2: Additional style anchor examples. Each pair shows the original reference (left) and the extracted anchor (right).
Figure A3: Degenerate style anchors. Ensemble Inversion may produce an anchor with no structure or one that copies the reference, especially when content and style are tightly coupled.
Figure A4: Artifacts in the generation stage. The adapter may misinterpret inversion noise in style anchor as semantic content, producing noisy lines and blobs.
Style-content dual-reference generation aims to synthesize an image that preserves the structure and semantics of a content reference while adopting the style of a separate style reference.Despite recent progress, this setting remains challenging because models must balance content fidelity, style alignment, and instruction following avoiding semantic leakage from the style reference.A key bottleneck is the lack of large-scale triplet data with clean content-style separation and broad long-tail style coverage.In this work, we propose FreeStyle, a scalable dual-reference generation framework based on community LoRA mining.We treat community LoRAs as compositional anchors for style and content, and design a rigorous generation and filtering pipeline to construct large-scale Style-Reference and Content-Reference triplets across multiple base models.To address content leakage, we adopt a two-stage curriculum with stage-specific disentanglement mechanisms: an attention-level enrichment constraint that suppresses style-reference leakage in the style-transfer stage, and a frequency-aware RoPE modulation strategy that targets positional-correspondence-based leakage in the harder dual-reference stage.We also introduce a benchmark covering both style-reference and dual-reference generation, with evaluations on style similarity, content preservation, aesthetics, instruction following, and leakage rejection. The benchmark incorporates a style-invariant Content Alignment Score (CAS) and introduces a calibrated VLM-based Rejection Score for evaluating generation reliability and leakage suppression.Extensive experiments show that our model achieves a strong balance among style alignment, content preservation, and leakage suppression.
Jinghong Lan, Wei Cheng, Yunuo Chen +10
Fudan University · StepFun · University of Hong Kong +1
Content-Preserving Style transfer, given content and style references, remains challenging for Diffusion Transformers (DiTs) due to entangled content and style features. With a reverse triplet synthesis pipeline to build a million-scale training set and a dual-branch Style-Content DiT (SC-DiT) that decouples style and content via separate ROPE embeddings and causal masking, we observe that such a one-stage training paradigm on mixed style categories causes semantic styles to dominate, hindering texture style learning, and harming content preservation. To address these issues, we propose Style-CCL, a Multi-Stage Curriculum Continual Learning framework that trains SC-DiT from semantic (easy) to texture (hard) styles, and from clean to synthetic data, with Random Memory Rehearsal across stages to avoid catastrophic forgetting. Extensive experiments demonstrate that our Style-CCL achieves state-of-the-art performance in three core metrics: style similarity, content consistency, and aesthetic quality.
Shiwen Zhang, Haoyuan Wang, Xianghao Zang +3
Institute of Artificial Intelligence (TeleAI), China Telecom.
Image-guided style transfer aims to apply the artistic characteristics of a style image to a content image while preserving its semantic structure and layout. Despite advances in diffusion-based methods, existing approaches often face challenges in disentangling content and style, particularly when independently optimized adapters are naively combined, causing conflicts between adapters and limiting controllability over the content-style balance in inference. We further demonstrate that training-free structural guidance directly derived from the content image through the internal attention of pre-trained model outperforms a dedicated content LoRA adapter in terms of structural fidelity and computational efficiency. Building on these observations, we propose AnyStyle, a streamlined framework for image-guided style transfer. The framework adopts a unified single-adapter paradigm for coherent style capture from the style image and incorporates training-free structural guidance from the content image, thus avoiding complex entanglement between multiple adapters and improving controllability and stability. Extensive experiments show that our method delivers competitive quantitative performance and significantly improved perceptual quality. Code is available at https://github.com/Yvan1001/AnyStyle.
Yongwen Lai, Chaoqun Wang
School of Artificial Intelligence, South China Normal University, Guangzhou, China