Facial stylization asks for a delicate transformation: the image should look unmistakably artistic, yet the person should remain unmistakably the same. Existing training-free and instruction-driven methods often fail to expose such control, either drifting away from identity or producing only weak stylization. We introduce Stylus, a training-free framework that treats facial style transfer as navigation on the curved latent manifold of a pretrained diffusion model. Rather than interpolating in flat Euclidean space, \method refines the interpolated content--style point by lowering a ratio-weighted discrete geodesic energy under the pullback metric, made tractable by a rank-one power-iteration approximation of the Jacobian. This principle drives two complementary controls: content--style latent interpolation for smooth semantic fusion, and self-attention query optimization for spatially adaptive style injection. The resulting dual-control mechanism provides continuous, fine-grained control over stylization strength while preserving identity-critical facial structure, all without additional training. Experiments show that Stylus achieves the strongest identity and content preservation at competitive stylization strength.
Figures & tables
Figure 1: Geometric interpretation of the gradient computation: the hidden-state map g maps points from coordinate space X to the manifold M , while the Jacobian Jg projects the manifold gradient onto the tangent space of X .
Figure 2: Visual comparison of style transfer results. Our Stylus better preserves facial identity while effectively transferring the artistic style compared to baseline methods.
WikiArt
MetFace
Method
ArtFID ( ↓ )
LPIPS ( ↓ )
ID ( ↓ )
CS ( ↑ )
ArtFID ( ↓ )
LPIPS ( ↓ )
ID ( ↓ )
CS ( ↑ )
AesPA-Net Hong et al. (2023)
28.3402
0.4076
0.2696
0.4254
29.1623
0.4853
0.2968
0.3948
ArtFlow An et al. (2021)
22.9778
0.4933
0.2555
0.3215
30.2701
0.4717
0.2484
0.3932
CAST Zhang et al. (2022)
33.0048
0.3856
0.2832
0.4064
28.6567
0.4109
0.2877
0.4293
StyTR 2 Deng et al. (2022)
24.2880
0.4068
0.1760
0.5181
26.6994
0.4104
0.1599
0.5503
InST Zhang et al. (2023b)
29.8916
0.5975
0.8075
0.1862
33.4454
0.6084
0.8432
0.2447
Table 1: The style transfer results with using WikiArt and MetFace datasets as style reference.
Figure 3: Style interpolation results showing smooth transitions between different artistic styles.
Figure 5
Figure 5: Discrete path-curvature statistics for linear, energy-refined, and randomly perturbed interpolation trajectories.
Figure 7Figure 8
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
k
ArtFID ( ↓ )
LPIPS ( ↓ )
ID ( ↓ )
Time (s/image)
1 (default)
26.6203
0.3118
0.1424
9.71
2
26.51
0.3103
0.1418
15.4
4
26.49
0.3099
0.1416
25.8
Appendix
Table 3: Rank-one vs. top- k singular directions (MetFace style references, SD v1.4). k=1 is the default setting of Stylus, identical to Table 1 .
Figure 8: More Visualization Samples of the proposed Stylus ∗ .
Figure 9: More Visualization Samples of the proposed Stylus ∗ .
Method
Style Faithfulness ↑
Identity Preservation ↑
Visual Quality ↑
Overall Preference ↑
StyTR 2 Deng et al. (2022)
3.62 ± 0.41
3.71 ± 0.38
3.68 ± 0.36
3.67 ± 0.39
StyleID Chung et al. (2024)
3.89 ± 0.36
4.02 ± 0.31
3.94 ± 0.34
3.96 ± 0.33
StyleSSP Xu et al. (2025)
3.74 ± 0.38
3.85 ± 0.33
3.79 ± 0.35
3.80 ± 0.34
Stylus (Ours)
4.31 ± 0.29
4.47 ± 0.24
4.28 ± 0.27
4.35 ± 0.26
Appendix
Table 4: User study results for the four evaluated methods (mean ± standard deviation, scale 1–5, higher is better).
Table 6: Style transfer results of StyleID and Stylus with different Stable Diffusion base models, using WikiArt and MetFace as style references.
Figure 10: The style transfer results of object stylization, scene stylization and art-to-art transfer.
Figure 16
Figure 13: Effect of the number of sampling steps (20 vs. 50) on style transfer performance, using MetFace as style references.
Figure 14: Linear baseline interpolation.
Figure 15: More visualization results showing style transfer examples with different content and style image pairs.
Figure 16: Qualitative comparison of style transfer results against DiT-based editing models.
Figure 17: Qualitative comparison of style transfer results against diffusion-based and conventional baselines.
Figure 18: Zoom-in visualization comparing original and stylized image regions. The detailed views highlight our method’s ability to preserve facial features while incorporating artistic elements.
Figure 19: Additional artistic-to-artistic style transfer results. Each row shows a different example with content image, style reference, and our stylization result.
Figure 20: Additional style transfer results using non-facial artistic references. Each row shows a different example with content image, non-facial style reference, and our stylization result.
Figure 21: Style transfer results using different versions of Stable Diffusion models.
Figure 22: Style transfer results using different versions of Stable Diffusion models.