SIC3D: Style Image Conditioned Text-to-3D Gaussian Splatting Generation

Ming He; Zhixiang Chen; Steve Maddock

cs.CVApr 9, 2026

SIC3D: Style Image Conditioned Text-to-3D Gaussian Splatting Generation

Authors: Ming He, Zhixiang Chen, Steve Maddock

Organizations: School of Computer Science, University of Sheffield

Abstract

Recent progress in text-to-3D object generation enables the synthesis of detailed geometry from text input by leveraging 2D diffusion models and differentiable 3D representations. However, the approaches often suffer from limited controllability and texture ambiguity due to the limitation of the text modality. To address this, we present SIC3D, a controllable image-conditioned text-to-3D generation pipeline with 3D Gaussian Splatting (3DGS). There are two stages in SIC3D. The first stage generates the 3D object content from text with a text-to-3DGS generation model. The second stage transfers style from a reference image to the 3DGS. Within this stylization stage, we introduce a novel Variational Stylized Score Distillation (VSSD) loss to effectively capture both global and local texture patterns while mitigating conflicts between geometry and appearance. A scaling regularization is further applied to prevent the emergence of artifacts and preserve the pattern from the style image. Extensive experiments demonstrate that SIC3D enhances geometric fidelity and style adherence, outperforming prior approaches in both qualitative and quantitative evaluations.

Figures & tables

Fig. 1: Overview of the proposed SIC3D framework. The pipeline consists of two stages. In the Object Generation Stage , Variational Score Distillation (VSD) is employed to produce a geometrically consistent 3D Gaussian Splatting representation from the text input. In the Style Distillation Stage , we introduce Variational Stylized Score Distillation (VSSD), which inject style features from the reference image into the diffusion model.

Fig. 2: Overview of the Style Distillation Stage. Starting from the first-stage result

O_{1}

, generated from text prompt

y

, two pre-trained diffusion models

\phi

augmented with IP-Adapter, LoRA and a camera projection layer are used to obtain the stylized object

O_{s}

. The style image

I_{s}

and text

y

are encoded by a CLIP encoder into

e_{I}

and

e_{y}

. IP-Adapter further processes

e_{I}

and injects style features into the diffusion model via cross-attention, while camera parameters are projected into

e_{c}

to condition the model on viewpoint. During optimization, only the 3DGS parameters, LoRA

\psi

, and the camera projection layer are updated using the VSSD loss

L_{\mathrm{VSSD}}

, the scaling loss

L_{\mathrm{surface}}

, and the diffusion loss

L_{\psi}

Fig. 3: Impact of different scales of Gaussians. The red solid line indicates the target boundary, while the black dashed line represents the object surface. In the top row, where the Gaussian scale is too large, significant artifacts appear, even when the Gaussian centers are near the target boundary, leading to blurry boundaries. In contrast, the bottom row shows that smaller scales lead to higher fidelity and smoother contours.

Fig. 4: Results from SIC3D. The object in the left column is generated with the prompt “An ancient Egyptian pyramid” The corresponding style images are shown in the top right of each sample. More visual results can be found in the Supplementary Material.

Fig. 5: Qualitative comparison between SIC3D, the stylized-prompt baseline, and state-of-the-art 3D style transfer methods (G-Style [ 10 ] , StyleGaussian [ 12 ] , SGSST [ 3 ] ). Each example shows three renderings from different viewpoints. More examples are given in the Supplementary Material.

Method	NNFM $\downarrow$	RMSE $\downarrow$	LPIPS $\downarrow$
Stylized Prompt	0.156	1.413	0.881
StyleGaussian	0.159	1.232	0.882
G-Style	0.154	1.390	0.881
SGSST	0.153	1.417	0.889
SIC3D	0.151	1.301	0.877

TABLE I: Quantitative comparison on masked renderings in terms of NNFM, RMSE, and LPIPS. Bold indicates the best performance for each metric.

Method	Object Quality	Style Alignment
Stylized Prompt	1044.600	878.905
StyleGaussian [ 12 ]	925.368	848.153
G-Style [ 10 ]	1015.814	893.972
SGSST [ 3 ]	983.604	980.170
SIC3D	1075.125	1154.929

TABLE II: GPT-4o evaluation (ELO scores) for Object Quality and Style Alignment across different stylized 3DGS generation methods. Higher is better; bold indicates the best score.

Fig. 6: Ablation on the scaling constraint. (a) With the constraint, Gaussians remain compact, leading to sharp boundaries and stable geometry. (b) Without it, Gaussians expand, causing artifacts and blurred contours. In each case, we show RGB rendering on the left and normal map on the right.

Fig. 7: Ablation on camera sampling. (a) Random viewpoints yield lower object quality and weaker surface style patterns. (b) Fixed viewpoints result in clearer geometry and more stable texture stylization.

Fig. 8: Ablation on LoRA. (a) Without LoRA, surface style pattern are insignificant. (b) With LoRA, surfaces reflect strong style patterns from

I_{s}

Setting	Stage 1	Stage 2
Ours (with LoRA)	18.7	18.9
Ours (without LoRA)	9.58	9.8

TABLE III: Estimated peak GPU memory usage (in GB) with and without LoRA.

Setting	Stage 1	Stage 2
Ours (with LoRA)	0.66	0.85
Ours (without LoRA)	0.38	0.59

TABLE IV: Estimated per-step runtime (in seconds) with and without LoRA.

Complexity Level	cloud	northern light	fire	seaweed	wave
Level 1	fluffy cloud	aurora borealis	dancing flames	sunlit seaweed	massive waterwave
Level 2	fluffy, soft clouds against a deep blue sky.	graceful aurora with soft, ethereal green hues	vivid, dynamic flames with smooth, flowing shapes	golden, wavy seaweed with soft underwater lighting	majestic wave with crisp, dynamic curling motion
Level 3	fluffy white cloud floating peacefully against a deep blue sky, evoking a sense of calm and openness.	mesmerizing aurora borealis with sweeping green and blue waves, dancing across a star-studded night sky.	dancing flames, with bright orange and yellow fire dancing and twisting	long, golden-brown seaweed strands floating underwater, illuminated by sunlight filtering through the water.	powerful ocean wave cresting with intense energy, cascading white foam over vivid blue water under a clear sky.

TABLE V: Different level of complexity for style image description.

Fig. 9: Generation results for baseline with different complexity level of stylized prompt (see Table V )

Fig. 10: Input for Style Alignment Evaluation. Left part result is produced from baseline style prompt, the right part is produced from SIC3D pipeline.

Fig. 11: Multi-view generation results of two examples (“A rabbit”, “A boat” and two different style images) with different numbers of optimization steps in Stage 2 (100, 200, 300, 400, and 500). Each row corresponds to one text prompt–style image pair, and each column shows the result at a specific step.

Fig. 12: Influence of IP-Adapter scale. First image is the rendering of first stage result $\mathrm{g}_{1}$ of prompt ”a dog”. Then we show the result of applying different value of IP-Adapter scale (within range [0.0, 1.0]) with the style image of fire.

Fig. 13: Influence of timestep sampling range. Row (a) represents the generation process with timestep sampled from 0 to 1. Row (b) represents the results for different the upper limit, from 0 to $t_{h}$ . $t_{h}$ range from 0.1 to 1.0. Row (c) are the results for different lower limit, from $t_{l}$ to 1.0. $t_{l}$ range from 0.0 to 0.9.

Method	NNFM $\downarrow$	RMSE $\downarrow$	LPIPS $\downarrow$
InstantStyle	0.160	1.418	0.886
Ours	0.151	1.301	0.877

TABLE VI: Quantitative comparison between InstantStyle layer selection strategy and our method on masked renderings.

Fig. 14: Comparison of stylization results using SD1.5 and SDXL as the backbone diffusion model. SIC3D produces consistent stylization quality across both backbones.

Fig. 15: Comparison results on different generation model in Stage 1.

Fig. 17: More comparison results between SIC3D and baselines with different style images.

Text-to-3D Generation

Text-to-3D generation creates 3D objects and scenes from language while balancing semantic alignment, geometry, appearance, and spatial coherence.

See all 64 papers

Dec 3, 2025

InStyle: Instant Appearance Stylization of 3D Shapes

3D stylization is central to game development, virtual reality, and digital arts, where the demand for diverse assets calls for scalable methods that support fast, high-fidelity manipulation. Existing text-to-3D stylization methods typically distill from 2D image editors, requiring time-intensive per-asset optimization and exhibiting multi-view inconsistency due to the limitations of current text-to-image models, which makes them impractical for large-scale production. In this paper, we introduce GaussianBlender, a pioneering feed-forward framework for text-driven 3D stylization that performs edits instantly at inference. Our method learns structured, disentangled latent spaces with controlled information sharing for geometry and appearance from spatially-grouped 3D Gaussians. A latent diffusion model then applies text-conditioned edits on these learned representations. Comprehensive evaluations show that GaussianBlender not only delivers instant, high-fidelity, geometry-preserving, multi-view consistent stylization, but also surpasses methods that require per-instance test-time optimization - unlocking practical, democratized 3D stylization at scale.

Melis Ocal, Xiaoyan Xing, Yue Li +3

University of Amsterdam · Bosch Center for AI

3D Shape Editing 3D Representation Learning

Ask PDF

Apr 4, 2026

M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting

Conventional 3D style transfer methods rely on a fixed reference image to apply artistic patterns to 3D scenes. However, in practical applications such as virtual or augmented reality, users often prefer more flexible inputs, including textual descriptions and diverse imagery. In this work, we introduce a novel real-time styling technique M2StyleGS to generate a sequence of precisely color-mapped views. It utilizes 3D Gaussian Splatting (3DGS) as a 3D presentation and multi-modality knowledge refined by CLIP as a reference style. M2StyleGS resolves the abnormal transformation issue by employing a precise feature alignment, namely subdivisive flow, it strengthens the projection of the mapped CLIP text-visual combination feature to the VGG style feature. In addition, we introduce observation loss, which assists in the stylized scene better matching the reference style during the generation, and suppression loss, which suppresses the offset of reference color information throughout the decoding process. By integrating these approaches, M2StyleGS can employ text or images as references to generate a set of style-enhanced novel views. Our experiments show that M2StyleGS achieves better visual quality and surpasses the previous work by up to 32.92% in terms of consistency.

Xingyu Miao, Xueqi Qiu, Haoran Duan +4

Department of Computer Science, Durham University, Durham, UK · Department of Automation, Tsinghua University, Beijing, China · Tencent Jarvis Lab, Shenzhen, China +1

Topic pending

Ask PDF

Mar 22, 2026

Text-Image Conditioned 3D Generation

High-quality 3D assets are essential for VR/AR, industrial design, and entertainment, motivating growing interest in generative models that create 3D content from user prompts. Most existing 3D generators, however, rely on a single conditioning modality: image-conditioned models achieve high visual fidelity by exploiting pixel-aligned cues but suffer from viewpoint bias when the input view is limited or ambiguous, while text-conditioned models provide broad semantic guidance yet lack low-level visual detail. This limits how users can express intent and raises a natural question: can these two modalities be combined for more flexible and faithful 3D generation? Our diagnostic study shows that even simple late fusion of text- and image-conditioned predictions outperforms single-modality models, revealing strong cross-modal complementarity. We therefore formalize Text-Image Conditioned 3D Generation, which requires joint reasoning over a visual exemplar and a textual specification. To address this task, we introduce TIGON, a minimalist dual-branch baseline with separate image- and text-conditioned backbones and lightweight cross-modal fusion. Extensive experiments show that text-image conditioning consistently improves over single-modality methods, highlighting complementary vision-language guidance as a promising direction for future 3D generation research. Project page: https://jumpat.github.io/tigon-page

Jiazhong Cen, Jiemin Fang, Sikuang Li +8

MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University · Huawei Inc. · Huazhong University of Science and Technology

3D Asset Generation 3D Shape Generation Multimodal Generation Text-to-3D Generation

Ask PDF

Abstract

Text-to-3D Generation

Explore similar work

InStyle: Instant Appearance Stylization of 3D Shapes

M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting

Text-Image Conditioned 3D Generation