FactorSplat: Appearance-Controllable Gaussian Proxies for Medical Volume Rendering

Zhongpai Gao; Benjamin Planche; Meng Zheng; Anwesa Choudhuri; Terrence Chen; Ziyan Wu

cs.CVOct 1, 2026

FactorSplat: Appearance-Controllable Gaussian Proxies for Medical Volume Rendering

Authors: Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Ziyan Wu

Organizations: United Imaging Intelligence, Boston MA, USA

Abstract

Transfer functions (TFs) control color and visibility in medical volume rendering, but image-trained Gaussian proxies typically bake one transfer function into their appearance. We present FactorSplat, a per-scene N-dimensional Gaussian splatting (N-DGS) proxy that accepts region-specific intensity-to-RGBA curves at inference. A local lookup applies the authored color and opacity change, while a shared functional encoder and low-rank per-Gaussian factors learn the residual appearance response. Geometry and directional appearance remain shared across presets, with visibility control and TF-aware pruning preserving the ability to hide and reveal structures. On seven CT and MR scans, FactorSplat improves mean PSNR and changed-region error over region-aware VEG across validation, interpolation, unseen composition, and out-of-distribution (OOD) edits. Across these four splits, seven-scan mean PSNR gains over VEG range from 1.10 to 1.52 dB. One checkpoint per scan supports unseen edits without retraining. At $1600^2$ , the cached fast renderer averages 524 FPS with 1.17 ms TF switches. Project page: https://gaozhongpai.github.io/FactorSplat/.

Figures & tables

Figure 1: Change appearance without retraining. Two vascular TF edits: highlight selected tissue (top) and change tissue colors (bottom). Full-frame PSNR.

Figure 2: Method overview. FactorSplat combines a physical lookup and low-rank residual over local material–intensity support to condition DC color and opacity.

		$\Delta_{\mathrm{TF}}\!\downarrow$ ( $\times 10^{-2}$ )				PSNR $\uparrow$				FPS $\uparrow$	Train (min) $\downarrow$
	Model	val	interp	comp	OOD	val	interp	comp	OOD	FPS $\uparrow$	Train (min) $\downarrow$
CT (5 scans)	N-DGS (identity floor)	3.60	6.49	8.96	17.74	31.11	27.88	25.50	21.11	851	6.9
	N-DGS (specialist)	—				30.70	30.82	31.09	31.94	866	92.8
	VEG	1.40	2.55	3.05	9.15	31.80	30.90	30.64	25.37	134	44.3
	VEG (native-quantized)	1.41	2.56	3.06	9.17	31.65	30.78	30.54	25.32	136	44.3
	FactorSplat (Ours)	1.36	2.17	2.88	8.09	33.08	32.29	31.33	26.58	508	10.5

Table 1: CT/MR quality and efficiency. Identity floor: no TF input; specialists: per-target-TF training. B200 FPS at

1600^{2}

; OOD: five edits. Per-scan results: Table 3 ; timing: Section 4.4 .

Figure 3: Held-out CT and MR examples. Heart (rows 1–3), vascular (4–5), nose (6), and hand (7). Specialists are trained separately on each target TF. Full-frame PSNR; bold marks each row’s best.

	heart				vascular				robustness
Variant	val	int.	cmp	OOD	val	int.	cmp	OOD	rank $\downarrow$	gap (dB) $\downarrow$
FactorSplat (Full)	30.79	29.33	28.68	23.77	34.20	33.68	32.59	30.88	1.69	0.48
$-$ exact gate	30.79	29.33	28.68	23.45	34.20	33.68	32.59	30.36	1.94	0.52
$-$ learned residual	30.16	28.69	27.61	23.22	33.70	32.55	31.92	29.28	3.62	1.60
$-$ physical lookup	29.77	27.48	26.11	20.30	33.31	32.13	29.94	29.18	4.88	3.47
$-$ functional code	30.76	28.38	28.42	22.75	34.30	33.68	33.07	28.04	2.88	2.84

Table 2: Two-scene ablations (

r{=}8

). PSNR

\uparrow

, mean rank, and worst gap over eight comparisons.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

(a) heart (CT, 12 labels)
	$\Delta_{\mathrm{TF}}\!\downarrow$ ( $\times 10^{-2}$ )				PSNR $\uparrow$
Model	val	interp	comp	OOD	val	interp	comp	OOD
N-DGS (identity floor)	3.74	6.76	10.28	19.92	28.31	25.18	21.49	17.77
N-DGS (specialist)	—				28.82	29.06	29.21	30.47
VEG	1.22	2.26	3.03	5.93	28.79	28.21	27.80	25.70
VEG (native-quantized)	1.22	2.26	3.03	5.95	28.69	28.13	27.73	25.66

Appendix

Table 3: Per-scan comparison. Changed-region

\Delta_{\mathrm{TF}}

(

\times 10^{-2}

) and PSNR; OOD averages five edits and (h) is the unweighted mean. Bold marks column best.

	heart				vascular
Variant	val	int.	cmp	OOD	val	int.	cmp	OOD
FactorSplat (Full)	1.31	2.63	3.14	9.64	1.60	2.48	3.59	4.54
$-$ exact gate	1.31	2.63	3.14	10.87	1.60	2.48	3.59	5.01
$-$ learned residual	1.39	2.87	3.62	10.56	1.67	3.40	4.26	6.83
$-$ physical lookup	2.86	5.27	6.59	14.71	2.95	4.67	6.98	7.77
$-$ functional code	1.35	3.46	3.17	10.92	1.50	2.43	3.17	7.85

Appendix

Table 4: Ablation edit fidelity. Changed-region

\Delta_{\mathrm{TF}}

(

\times 10^{-2}

; lower is better).

Scene	Ours	VEG	Ours	VEG	Ours	VEG	Speedup
	$N$ (k)		PSNR $\uparrow$		FPS $\uparrow$
heart	278.6	146.3	27.89	27.51	477	109	4.36 $\times$
vascular	235.7	148.1	32.72	29.47	525	181	2.90 $\times$
intestine	281.8	208.9	29.77	28.57	494	122	4.06 $\times$
lower	250.5	127.0	32.73	31.36	531	155	3.42 $\times$
kneejoint	80.4	82.7	29.75	30.21	515	103	5.03 $\times$

Appendix

Table 5: B200 deployment at

1600^{2}

. Matched views; FactorSplat FPS uses cached fast rendering. PSNR uses quality rendering.

Figure 4: Additional held-out CT examples. Vascular (rows 1–3), heart (4–5), intestine (6), and lower body (7). Full-frame PSNR; bold marks each row’s best.

3D Gaussian Splatting

3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.

See all 1,019 papers

Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering
May 22, 2025Zhongpai Gao, Benjamin Planche, Meng Zheng +4Neural Rendering 3D Medical Imaging
United Imaging Intelligence, Boston, MA, USA
XClipGS: Exact Half-Space Clipping for Medical Volume Gaussian Splatting
Aug 7, 2026Zhongpai Gao, Benjamin Planche, Meng Zheng +43D Gaussian Splatting 3D Medical Imaging
United Imaging Intelligence, Boston MA, USA · Clemson University, Clemson SC, USA
CTBakeGS: training-free baking of photorealistic CT volume renderings into 3D Gaussian splats
Oct 27, 2026Haruki Fukuda, Constantin Lux, Wolf Schweitzer +23D Gaussian Splatting 3D Medical Imaging
Department of Forensic Medicine, Gunma University Graduate School of Medicine, Maebashi, Japan · Institute of Forensic Medicine, Universität Zürich, Zürich, Switzerland · Biometrics/Forensic Imaging, Zurich Forensic Science Institute, Zurich, Switzerland

May 22, 2025J/K move · Enter open · S save

Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering

Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri+3

United Imaging Intelligence, Boston, MA, USA

Neural Rendering 3D Medical Imaging Gaussian Splatting Computed Tomography

Open paper AskPDF arXiv

Photorealistic volumetric rendering of CT scans greatly benefits clinical workflows, yet neural approaches such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) require prohibitive per-scan optimization (hours for NeRF, about 30 minutes for 3DGS), making them impractical in clinical settings. We propose Render-FM, a feedforward model that eliminates this bottleneck by directly regressing 6D Gaussian Splatting (6DGS) parameters from a CT volume in a single 2.8-second forward pass, a 500x speedup over per-scan optimization. To bridge the domain gap between natural scene reconstruction and medical volumetric rendering, we introduce Anatomy-Guided Priming (AGP), which incorporates segmentation masks and transfer functions as structural and appearance priors, information that existing Gaussian splatting methods overlook. Built on an nnU-Net-inspired 3D U-Net trained on diverse CT scans, Render-FM predicts per-voxel 6DGS parameters and supports immediate real-time rendering. Unlike per-scan methods, it generalizes to unseen anatomies, novel transfer functions, and enables compositional organ visualization with zero additional preparation time. Optional 89-second fine-tuning further improves quality, surpassing per-scan optimized baselines. Project page: https://gaozhongpai.github.io/renderfm/.

Abstract

3D Gaussian Splatting

Explore similar work

Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering

XClipGS: Exact Half-Space Clipping for Medical Volume Gaussian Splatting

CTBakeGS: training-free baking of photorealistic CT volume renderings into 3D Gaussian splats