M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting
Authors: Xingyu Miao, Xueqi Qiu, Haoran Duan, Yawen Huang, Xian Wu, Jingjing Deng, Yang Long
Organizations: Department of Computer Science, Durham University, Durham, UK · Department of Automation, Tsinghua University, Beijing, China · Tencent Jarvis Lab, Shenzhen, China · School of Engineering Mathematics and Technology, Bristol University, Bristol, UK
Conventional 3D style transfer methods rely on a fixed reference image to apply artistic patterns to 3D scenes. However, in practical applications such as virtual or augmented reality, users often prefer more flexible inputs, including textual descriptions and diverse imagery. In this work, we introduce a novel real-time styling technique M2StyleGS to generate a sequence of precisely color-mapped views. It utilizes 3D Gaussian Splatting (3DGS) as a 3D presentation and multi-modality knowledge refined by CLIP as a reference style. M2StyleGS resolves the abnormal transformation issue by employing a precise feature alignment, namely subdivisive flow, it strengthens the projection of the mapped CLIP text-visual combination feature to the VGG style feature. In addition, we introduce observation loss, which assists in the stylized scene better matching the reference style during the generation, and suppression loss, which suppresses the offset of reference color information throughout the decoding process. By integrating these approaches, M2StyleGS can employ text or images as references to generate a set of style-enhanced novel views. Our experiments show that M2StyleGS achieves better visual quality and surpasses the previous work by up to 32.92% in terms of consistency.
Figures & tables
Figure 1: Multi-modality 3D style transfer with M 2 StyleGS. Employing a set of 3D scene images captured from various perspectives, M 2 StyleGS can effectively apply reference styles described in arbitrary images or text.
Figure 2: The pipeline of M 2 StyleGS. During the training phase, M 2 StyleGS performs style transfer using style images. After training, M 2 StyleGS is capable of applying multi-modality stylistic transformations directly using either images or text (the purple dashed arrows illustrate text feature processing.)
Figure 3: Comparison of the previous alignment method with the proposed subdivisive flow. (a) illustrates that the direct use of the mapping module can result in mismatches in feature projection. (b) illustrates the subdivisive flow learns the ODE that characterizes the trajectory from the CLIP feature flow to the VGG feature space precisely.
Figure 4: Qualitative comparison with SOTA 3D style transfer methods. M2StyleGS can achieve precise style transfer based on the reference style images or reference style text.
Figure 5: Ablation studies on subdivisive flow. The top row shows the r -th subdivisve round outcomes and their feature distributions among image style transfer. The bottom row shows the corresponding text style transfer, i.e. , the original text feature distribution is the same as the style image feature distribution. Hence, the initial SIM and FID are the same.
Image reference
Long-range
Short-range
Text reference
Long-range
Short-range
LPIPS
RMSE
LPIPS
RMSE
LPIPS
RMSE
LPIPS
RMSE
StyleRF
0.142
0.120
0.074
0.073
CLIPStyler
0.215
0.155
0.143
0.152
StyleGaussian
0.107
0.114
0.061
0.067
CLIPNeRF
0.131
0.106
0.088
0.079
ConRF
0.123
0.100
0.082
0.070
ConRF
0.144
0.128
0.075
0.080
Ours
0.099
0.089
0.055
0.052
Ours
0.120
0.102
0.070
0.057
Table 1: Consistency results. We compared M2StyleGS with the SOTA methods on consistency using RMSE( ↓ ) and LPIPS( ↓ ) within the LLFF dataset. We employed adjacent views to obtain short-range consistency and distant views to obtain long-range consistency. Note that in all the tables, the best score is in bold , and the second score is underline .
Image reference
Long-range
Short-range
Text reference
Long-range
Short-range
LPIPS
RMSE
LPIPS
RMSE
LPIPS
RMSE
LPIPS
RMSE
Ours w/o Lobs & Lsup
0.112
0.101
0.076
0.067
Ours w/o Lobs & Lsup
0.127
0.136
0.077
0.065
Ours w/o Lobs
0.109
0.097
0.062
0.056
Ours w/o Lobs
0.122
0.110
0.074
0.061
Ours w/o Lsup
0.105
0.095
0.069
0.058
Ours w/o Lsup
0.128
0.119
0.071
0.064
Ours
0.099
0.089
0.055
0.052
Ours
0.120
0.102
0.070
0.057
Table 2: Ablation studies with different loss functions on consistency. We explored the effect of different loss on the consistency.
Figure 6: Ablation study with different loss functions. (a): full system. (b): stylization without Lobs . (c): stylization without Lsup . (d): stylization without Lobs and Lsup .
While 3D Gaussian Splatting (3DGS) provides an efficient and explicit representation for novel view synthesis, enforcing stylistic coherence across viewpoints remains challenging. Existing 3D stylization methods typically apply 2D feature-matching losses independently per rendered view, which leads to unstable style allocation, many-to-one feature reuse, and limited cross-view consistency. We propose a capacity-controlled framework for multi-view stylization of 3DGS, grounded in optimal transport. Specifically, we reformulate local style matching as a semi-balanced optimal transport problem. By introducing explicit column-capacity constraints with tunable strength, our formulation mitigates many-to-one matching and enables controllable allocation of style features. This transport-based objective provides a principled mechanism for balancing feature coverage and stylistic diversity while maintaining stable correspondences across viewpoints. To further enhance cross-view coherence, we incorporate a novel cross-view matching guidance to constrain correspondences between scene content and style patterns. In addition, we introduce several geometric regularizations to enhance the vanilla 3DGS, thereby enabling optimized Gaussian primitives to represent finer-grained textures during stylization. Extensive experiments demonstrate that our approach significantly improves multi-view stylistic consistency and produces stable, expressive 3D stylizations while preserving the core semantic structure of the scene.
Zhihao Wen, Yixin Yang, Bojian Wu +4
Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, CSSE, Shenzhen University, China · Tencent Games, China · The Hebrew University of Jerusalem, Israel +1
Recent progress in text-to-3D object generation enables the synthesis of detailed geometry from text input by leveraging 2D diffusion models and differentiable 3D representations. However, the approaches often suffer from limited controllability and texture ambiguity due to the limitation of the text modality. To address this, we present SIC3D, a controllable image-conditioned text-to-3D generation pipeline with 3D Gaussian Splatting (3DGS). There are two stages in SIC3D. The first stage generates the 3D object content from text with a text-to-3DGS generation model. The second stage transfers style from a reference image to the 3DGS. Within this stylization stage, we introduce a novel Variational Stylized Score Distillation (VSSD) loss to effectively capture both global and local texture patterns while mitigating conflicts between geometry and appearance. A scaling regularization is further applied to prevent the emergence of artifacts and preserve the pattern from the style image. Extensive experiments demonstrate that SIC3D enhances geometric fidelity and style adherence, outperforming prior approaches in both qualitative and quantitative evaluations.
Ming He, Zhixiang Chen, Steve Maddock
School of Computer Science, University of Sheffield
The growing demand for rapid and scalable 3D asset creation has driven interest in feed-forward 3D reconstruction methods, with 3D Gaussian Splatting (3DGS) emerging as an effective scene representation. While recent approaches have demonstrated pose-free reconstruction from unposed image collections, integrating stylization or appearance control into such pipelines remains underexplored. Existing attempts largely rely on image-based conditioning, which limits both controllability and flexibility. In this work, we introduce AnyStyle, a feed-forward 3D reconstruction and stylization framework that enables pose-free, zero-shot stylization through multimodal conditioning. Our method supports both textual and visual style inputs, allowing users to control the scene appearance using natural language descriptions or reference images. We propose a modular stylization architecture that requires only minimal architectural modifications and can be integrated into existing feed-forward 3D reconstruction backbones. Experiments demonstrate that AnyStyle improves style controllability over prior feed-forward stylization methods while preserving high-quality geometric reconstruction. A user study further confirms that AnyStyle achieves superior stylization quality compared to an existing state-of-the-art approach. Repository: https://github.com/joaxkal/AnyStyle.
Joanna Kaleta, Bartosz Świrta, Kacper Kania +3
Warsaw University of Technology · Sano Centre for Computational Medicine · IDEAS NCBR +2