Authors: Min Lu, Yuanfeng He, Anthony Chen, Jianhuang He, Pu Wang, Daniel Cohen-Or, Hui Huang
Organizations: Shenzhen University, China · Visual Computing Research Center (VCC), College of Computer Science and Software Engineering (CSSE), Shenzhen University, China · Peking University, China
Artistic styles often embed abstraction beyond surface appearance, involving deliberate reinterpretation of structure rather than mere changes in texture or color. Conventional style transfer methods typically preserve the input geometry and therefore struggle to capture this deeper abstraction behavior, especially for illustrative and nonphotorealistic styles. In this work, we introduce Abstraction in Style (AiS), a generative framework that separates structural abstraction from visual stylization. Given a target image and a small set of style exemplars, AiS first derives an intermediate abstraction proxy that reinterprets the target's structure in accordance with the abstraction logic exhibited by the style. The proxy captures semantic structure while relaxing geometric fidelity, enabling subsequent stylization to operate on an abstracted representation rather than the original image. In a second stage, the abstraction proxy is rendered to produce the final stylized output, preserving visual coherence with the reference style. Both stages are implemented using a shared image space analogy, enabling transformations to be learned from visual exemplars without explicit geometric supervision. By decoupling abstraction from appearance and treating abstraction as an explicit, transferable process, AiS supports a wider range of stylistic transformations, improves controllability, and enables more expressive stylization.
Figure 2. Conventional style transfer keeps the structure of the target image. Introducing an abstraction stage prior to the stylization, allows adopting structural attributes from the reference, leading to a more coherent transformation.
Figure 3. Two stages of AiS: in the abstraction stage, the target image is reshaped following the reference, generating an abstraction proxy ; Then, in the second stage, the proxy is stylized to produce the final output.
Figure 4. Structural Abstraction Process: given a target image, its hidden backbone is first computed via skeleton detection and area erosion. An abstraction proxy is then generated by the A-VAT, which is trained to learn the visual analogy from the Backbone → Proxy pairs in the reference set, e.g., the pair of roosters.
Figure 5. Hidden Backbone Construction: four examples are shown. For each example, the process proceeds as follows: the input target image (left) is first vectorized into simplified flat-color shapes (middle); the skeleton is then detected, and regions are eroded to yield the final hidden structure (right).
Figure 6. Examples of Abstraction Proxies Inferred by A-VATs: for the same hidden backbone in each row, distinct abstraction proxies are generated by different A-VATs which are trained on different reference respectively.
Figure 7. Visual Analogy Transfer (VAT) realized with DiT and LoRA: A visual transformation is expressed as a 2×2 analogy, where a reference pair ( A→A′ ) defines a relation that is applied to a new input ( B ) by predicting the missing panel ( B′ ). VAT is implemented using a Diffusion Transformer (DiT) conditioned on the visible panels, with a lightweight LoRA used to adapt the model to the analogy relation.
Figure 8. Mix of Three Different A-VATs with Four S-VATs: giving the same S-VAT, the generated outputs share visual appearance (color, texture) but exhibit distinct geometry, demonstrating independent control of style and structure.
Figure 9. Generated examples in which textual elements are stylized with the primary visual content.
Figure 10. Ablation of the Eroded Region Cues in Hidden Backbone: skeleton-only representation (i.e., without eroded regions) can produce a geometrically degraded abstraction proxy.
Figure 11. Ablation of the Abstraction Stage: the two ablated framework variants, without Abstraction and with only Vector Simplification , fail to generalize visual logic. Their outputs exhibit geometry that strictly adheres to the original input target.
Figure 12. Comparison of the two-stage (A-VAT + S-VAT) and a distilled single-stage variant (AS-VAT). The single-stage AS-VAT fails to synthesize coherent results, yielding visual artifacts such as "holes," color bleeding, and rigid adherence to input geometry.
Metrics
StyleID
A.D.
StyleA.
B-LoRA
ZipLoRA
CSD
0.36
0.54
0.22
0.37
0.34
Style FID
529.05
431.12
511.59
495.20
547.73
CLIP-I
0.62
0.69
0.61
0.63
0.66
K-LoRA
Nano Ban.
AiS (Ablation Study)
w/o Abs.
+ Vec. Simpl.
AiS (Full)
CSD
0.25
0.56
0.46
0.66
0.75 ↑
Table 1. Quantitative comparison of baseline methods and ablations.
Figure 13. Generated examples: For the target image in each row, its style-agnostic hidden backbone is first generated. Then, five stylized outputs are produced, one for each reference style (columns). For each style, the system uses a trained A-VAT to generate an abstraction proxy (left grayscale image in each pair), which is then stylized via the corresponding S-VAT to create the final result (right image in the pair).
Figure 14. Generated examples: Each column corresponds to a different target image. Each row corresponds to a different reference style (with one exemplar shown per style), which is applied to all targets in that row. The full set for each style comprises 5 to 10 exemplars, available in the supplementary material.
Figure 15. Design examples mixing up the reference exemplars and the results generated by AiS: on the side are the original images from which the results are generated. The original images are ordered according to the result positions in the example, from top to bottom, left to right.
Figure 16. Qualitative comparison with baseline methods: existing methods struggle with abstract styles and over-rigidly preserve input structure, producing unsatisfactory results. Our method excels at capturing nuanced artistic styles while maintaining natural structural variations. A larger version of this figure, including additional examples, is provided in the supplementary material.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 17. Reference exemplars and 2×2 image training samples for A-VAT ( Backbone → Proxy ) and S-VAT ( Proxy → Output).
Figure 18. Ablation of the 2×2 Analogy Layout Design: training S-VAT with 1×2 image composite fails to learn the effective style transferring.
Figure 19. Ablation of the Grayscale Proxy Design: In Color S-VAT (color proxy → output), the color from the proxy is effectively transferred to the output. In contrast, S-VAT (grayscale proxy → output) infers new colors for the proxy that align with the reference.
Figure 20. Full sets of exemplars in sampled references: for each reference style, it shows the number of exemplars in the set (# Exemplars) and the number of training images for VAT (# Training Images).
Figure 21. Gallery of generated examples (part I).
Figure 22. Gallery of generated examples (part II).
Figure 24. Evaluation tests with baseline methods (part I).
Figure 25. Evaluation tests with baseline methods (part II).
Figure 26. Generated consistent design sets from diverse inputs (part I).
Figure 27. Generated consistent design sets from diverse inputs (part II).
Diffusion-based stylization has advanced significantly, yet existing methods are limited to color-driven transformations, neglecting complex semantics and material details. We introduce StyleExpert, a semantic-aware framework based on the Mixture of Experts (MoE). Our framework employs a unified style encoder, trained on our large-scale dataset of content-style-stylized triplets, to embed diverse styles into a consistent latent space. This embedding is then used to condition a similarity-aware gating mechanism, which dynamically routes styles to specialized experts within the MoE architecture. Leveraging this MoE architecture, our method adeptly handles diverse styles spanning multiple semantic levels, from shallow textures to deep semantics. Extensive experiments show that StyleExpert outperforms existing approaches in preserving semantics and material details, while generalizing to unseen styles. Our code and collected images are available at the project page: https://hh-lg.github.io/StyleExpert-Page/.
Shihao Zhu, Ziheng Ouyang, Yijia Kang +5
VCIP, CS, Nankai University · Tianjin University · vivo BlueImage Lab +1
Despite the advancements in diffusion-based image style transfer, existing methods are commonly limited by 1) semantic gap: the style reference could miss proper content semantics, causing uncontrollable stylization; 2) reliance on extra constraints (e.g., semantic masks) restricting applicability; 3) rigid feature associations lacking adaptive global-local alignment, failing to balance fine-grained stylization and global content preservation. These limitations, particularly the inability to flexibly leverage style inputs, fundamentally restrict style transfer in terms of personalization, accuracy, and adaptability. To address these, we propose StyleGallery, a training-free and semantic-aware framework that supports arbitrary reference images as input and enables effective personalized customization. It comprises three core stages: semantic region segmentation (adaptive clustering on latent diffusion features to divide regions without extra inputs); clustered region matching (block filtering on extracted features for precise alignment); and style transfer optimization (energy function-guided diffusion sampling with regional style loss to optimize stylization). Experiments on our introduced benchmark demonstrate that StyleGallery outperforms state-of-the-art methods in content structure preservation, regional stylization, interpretability, and personalized customization, particularly when leveraging multiple style references.
Boyu He, Yunfan Ye, Chang Liu +3
College of Computer Science and Technology, National University of Defense Technology · School of Design, Hunan University
Diffusion models represent one of the most advanced paradigms in generative modeling. Leveraging their development, a growing number of style transfer methods based on diffusion models have been proposed. However, among these methods, multi-image style transfer approaches that require at least five to ten style examples tend to achieve more satisfactory results. Single-image methods, by contrast, often struggle with either insufficient content preservation or inadequate style fidelity. This greatly limits style extraction from scarce artworks and undermines their artistic value. To address this, we propose Abstract-LoRA, a method that pushes the boundaries of single-image style transfer through lightweight LoRA training on specific U-Net blocks in diffusion models. Specifically, our work is inspired by B-LoRA, a style transfer method that achieves basic style-content disentanglement by training specific U-Net blocks. However, it suffers from a critical limitation: the inability to capture complex backgrounds. Building upon B-LoRA, our method conducts a more refined analysis of U-Net blocks, employing additional U-Net blocks and clustering-based abstraction of style images to better disentangle and balance style and content. Extensive experiments demonstrate that our proposed method not only generates visually more harmonious and satisfying artistic images but also quantitatively improves the preservation of both style and content in the final outputs.
Xinglin Hu
School of Data Science, The Chinese University of Hong Kong, Shenzhen