SIC3D: Style Image Conditioned Text-to-3D Gaussian Splatting Generation

Ming He; Zhixiang Chen; Steve Maddock

cs.CVApr 9, 2026

SIC3D: Style Image Conditioned Text-to-3D Gaussian Splatting Generation

Authors: Ming He, Zhixiang Chen, Steve Maddock

Organizations: School of Computer Science, University of Sheffield

Abstract

Recent progress in text-to-3D object generation enables the synthesis of detailed geometry from text input by leveraging 2D diffusion models and differentiable 3D representations. However, the approaches often suffer from limited controllability and texture ambiguity due to the limitation of the text modality. To address this, we present SIC3D, a controllable image-conditioned text-to-3D generation pipeline with 3D Gaussian Splatting (3DGS). There are two stages in SIC3D. The first stage generates the 3D object content from text with a text-to-3DGS generation model. The second stage transfers style from a reference image to the 3DGS. Within this stylization stage, we introduce a novel Variational Stylized Score Distillation (VSSD) loss to effectively capture both global and local texture patterns while mitigating conflicts between geometry and appearance. A scaling regularization is further applied to prevent the emergence of artifacts and preserve the pattern from the style image. Extensive experiments demonstrate that SIC3D enhances geometric fidelity and style adherence, outperforming prior approaches in both qualitative and quantitative evaluations.

Figures & tables

Fig. 1: Overview of the proposed SIC3D framework. The pipeline consists of two stages. In the Object Generation Stage , Variational Score Distillation (VSD) is employed to produce a geometrically consistent 3D Gaussian Splatting representation from the text input. In the Style Distillation Stage , we introduce Variational Stylized Score Distillation (VSSD), which inject style features from the reference image into the diffusion model.

Fig. 2: Overview of the Style Distillation Stage. Starting from the first-stage result

O_{1}

, generated from text prompt

y

, two pre-trained diffusion models

\phi

augmented with IP-Adapter, LoRA and a camera projection layer are used to obtain the stylized object

O_{s}

. The style image

I_{s}

and text

y

are encoded by a CLIP encoder into

e_{I}

and

e_{y}

. IP-Adapter further processes

e_{I}

and injects style features into the diffusion model via cross-attention, while camera parameters are projected into

e_{c}

to condition the model on viewpoint. During optimization, only the 3DGS parameters, LoRA

\psi

, and the camera projection layer are updated using the VSSD loss

L_{\mathrm{VSSD}}

, the scaling loss

L_{\mathrm{surface}}

, and the diffusion loss

L_{\psi}

Fig. 3: Impact of different scales of Gaussians. The red solid line indicates the target boundary, while the black dashed line represents the object surface. In the top row, where the Gaussian scale is too large, significant artifacts appear, even when the Gaussian centers are near the target boundary, leading to blurry boundaries. In contrast, the bottom row shows that smaller scales lead to higher fidelity and smoother contours.

Fig. 4: Results from SIC3D. The object in the left column is generated with the prompt “An ancient Egyptian pyramid” The corresponding style images are shown in the top right of each sample. More visual results can be found in the Supplementary Material.

Fig. 5: Qualitative comparison between SIC3D, the stylized-prompt baseline, and state-of-the-art 3D style transfer methods (G-Style [ 10 ] , StyleGaussian [ 12 ] , SGSST [ 3 ] ). Each example shows three renderings from different viewpoints. More examples are given in the Supplementary Material.

Method	NNFM $\downarrow$	RMSE $\downarrow$	LPIPS $\downarrow$
Stylized Prompt	0.156	1.413	0.881
StyleGaussian	0.159	1.232	0.882
G-Style	0.154	1.390	0.881
SGSST	0.153	1.417	0.889
SIC3D	0.151	1.301	0.877

TABLE I: Quantitative comparison on masked renderings in terms of NNFM, RMSE, and LPIPS. Bold indicates the best performance for each metric.

Method	Object Quality	Style Alignment
Stylized Prompt	1044.600	878.905
StyleGaussian [ 12 ]	925.368	848.153
G-Style [ 10 ]	1015.814	893.972
SGSST [ 3 ]	983.604	980.170
SIC3D	1075.125	1154.929

TABLE II: GPT-4o evaluation (ELO scores) for Object Quality and Style Alignment across different stylized 3DGS generation methods. Higher is better; bold indicates the best score.

Fig. 6: Ablation on the scaling constraint. (a) With the constraint, Gaussians remain compact, leading to sharp boundaries and stable geometry. (b) Without it, Gaussians expand, causing artifacts and blurred contours. In each case, we show RGB rendering on the left and normal map on the right.

Fig. 7: Ablation on camera sampling. (a) Random viewpoints yield lower object quality and weaker surface style patterns. (b) Fixed viewpoints result in clearer geometry and more stable texture stylization.

Fig. 8: Ablation on LoRA. (a) Without LoRA, surface style pattern are insignificant. (b) With LoRA, surfaces reflect strong style patterns from

I_{s}

Setting	Stage 1	Stage 2
Ours (with LoRA)	18.7	18.9
Ours (without LoRA)	9.58	9.8

TABLE III: Estimated peak GPU memory usage (in GB) with and without LoRA.

Setting	Stage 1	Stage 2
Ours (with LoRA)	0.66	0.85
Ours (without LoRA)	0.38	0.59

TABLE IV: Estimated per-step runtime (in seconds) with and without LoRA.

Complexity Level	cloud	northern light	fire	seaweed	wave
Level 1	fluffy cloud	aurora borealis	dancing flames	sunlit seaweed	massive waterwave
Level 2	fluffy, soft clouds against a deep blue sky.	graceful aurora with soft, ethereal green hues	vivid, dynamic flames with smooth, flowing shapes	golden, wavy seaweed with soft underwater lighting	majestic wave with crisp, dynamic curling motion
Level 3	fluffy white cloud floating peacefully against a deep blue sky, evoking a sense of calm and openness.	mesmerizing aurora borealis with sweeping green and blue waves, dancing across a star-studded night sky.	dancing flames, with bright orange and yellow fire dancing and twisting	long, golden-brown seaweed strands floating underwater, illuminated by sunlight filtering through the water.	powerful ocean wave cresting with intense energy, cascading white foam over vivid blue water under a clear sky.

TABLE V: Different level of complexity for style image description.

Fig. 9: Generation results for baseline with different complexity level of stylized prompt (see Table V )

Fig. 10: Input for Style Alignment Evaluation. Left part result is produced from baseline style prompt, the right part is produced from SIC3D pipeline.

Fig. 11: Multi-view generation results of two examples (“A rabbit”, “A boat” and two different style images) with different numbers of optimization steps in Stage 2 (100, 200, 300, 400, and 500). Each row corresponds to one text prompt–style image pair, and each column shows the result at a specific step.

Fig. 12: Influence of IP-Adapter scale. First image is the rendering of first stage result $\mathrm{g}_{1}$ of prompt ”a dog”. Then we show the result of applying different value of IP-Adapter scale (within range [0.0, 1.0]) with the style image of fire.

Fig. 13: Influence of timestep sampling range. Row (a) represents the generation process with timestep sampled from 0 to 1. Row (b) represents the results for different the upper limit, from 0 to $t_{h}$ . $t_{h}$ range from 0.1 to 1.0. Row (c) are the results for different lower limit, from $t_{l}$ to 1.0. $t_{l}$ range from 0.0 to 0.9.

Method	NNFM $\downarrow$	RMSE $\downarrow$	LPIPS $\downarrow$
InstantStyle	0.160	1.418	0.886
Ours	0.151	1.301	0.877

TABLE VI: Quantitative comparison between InstantStyle layer selection strategy and our method on masked renderings.

Fig. 14: Comparison of stylization results using SD1.5 and SDXL as the backbone diffusion model. SIC3D produces consistent stylization quality across both backbones.

Fig. 15: Comparison results on different generation model in Stage 1.

Fig. 17: More comparison results between SIC3D and baselines with different style images.

Text-to-3D Generation

Text-to-3D generation creates 3D objects and scenes from language while balancing semantic alignment, geometry, appearance, and spatial coherence.

See all 64 papers

InStyle: Instant Appearance Stylization of 3D Shapes
Dec 3, 2025Melis Ocal, Xiaoyan Xing, Yue Li +33D Shape Editing 3D Representation Learning
University of Amsterdam · Bosch Center for AI
M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting
Apr 4, 2026Xingyu Miao, Xueqi Qiu, Haoran Duan +4
Department of Computer Science, Durham University, Durham, UK · Department of Automation, Tsinghua University, Beijing, China · Tencent Jarvis Lab, Shenzhen, China +1
Text-Image Conditioned 3D Generation
Mar 22, 2026Jiazhong Cen, Jiemin Fang, Sikuang Li +83D Asset Generation 3D Shape Generation
MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University · Huawei Inc. · Huazhong University of Science and Technology

Dec 3, 2025J/K move · Enter open · S save

InStyle: Instant Appearance Stylization of 3D Shapes

Melis Ocal, Xiaoyan Xing, Yue Li, Ngo Anh Vien+2

University of Amsterdam · Bosch Center for AI

3D Shape Editing 3D Representation Learning

Open paper AskPDF arXiv

3D stylization is central to game development, virtual reality, and digital arts, where the demand for diverse assets calls for scalable methods that support fast, high-fidelity manipulation. Existing text-to-3D stylization methods typically distill from 2D image editors, requiring time-intensive per-asset optimization and exhibiting multi-view inconsistency due to the limitations of current text-to-image models, which makes them impractical for large-scale production. In this paper, we introduce GaussianBlender, a pioneering feed-forward framework for text-driven 3D stylization that performs edits instantly at inference. Our method learns structured, disentangled latent spaces with controlled information sharing for geometry and appearance from spatially-grouped 3D Gaussians. A latent diffusion model then applies text-conditioned edits on these learned representations. Comprehensive evaluations show that GaussianBlender not only delivers instant, high-fidelity, geometry-preserving, multi-view consistent stylization, but also surpasses methods that require per-instance test-time optimization - unlocking practical, democratized 3D stylization at scale.

Abstract

Text-to-3D Generation

Explore similar work

InStyle: Instant Appearance Stylization of 3D Shapes

M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting

Text-Image Conditioned 3D Generation