CTBakeGS: training-free baking of photorealistic CT volume renderings into 3D Gaussian splats

Haruki Fukuda; Constantin Lux; Wolf Schweitzer; Tadashi Hosoya; Lars Ebert

cs.CVOct 27, 2026

CTBakeGS: training-free baking of photorealistic CT volume renderings into 3D Gaussian splats

Authors: Haruki Fukuda, Constantin Lux, Wolf Schweitzer, Tadashi Hosoya, Lars Ebert

Organizations: Department of Forensic Medicine, Gunma University Graduate School of Medicine, Maebashi, Japan · Institute of Forensic Medicine, Universität Zürich, Zürich, Switzerland · Biometrics/Forensic Imaging, Zurich Forensic Science Institute, Zurich, Switzerland

Abstract

Photorealistic volume rendering of computed tomography (CT) may convey anatomy intuitively, but it needs the CT data, dedicated software and a powerful graphics processor. Three-dimensional Gaussian splatting (3DGS) displays photorealistic scenes in real time on less powerful devices, but existing medical approaches train each scene for minutes to hours. We aimed to convert a CT volume rendering directly into a 3DGS model without training. Splats were placed on an adaptive octree of millimetre-sized cells. Their colour and opacity were computed on the graphics processor with the renderer's lighting, cell opacity was converted analytically into splat opacity, and faint splats were retained stochastically above the 1/255 opacity floor of standard 3DGS rasterisers. Six public CT scans (1.5-mm voxels) were converted with three transfer functions and compared with converged references by peak signal-to-noise ratio (PSNR), structural similarity and a learned perceptual metric. Two-million-splat models reached a PSNR of 32.6 +/- 0.9 dB for bone, 30.1 +/- 1.0 dB for contrast-enhanced vessels and 32.3 +/- 1.5 dB for skin and bone. A conversion pass (usually two or three per model) took 4-30 s on a current laptop and 9-60 s on a desktop with a 4-GB graphics card. On the laptop, the models rendered at 68 frames per second at 800 x 800 pixels. Training-free conversion produced real-time 3DGS models of photorealistic CT renderings in about a minute or less per conversion pass on ordinary computers, with fidelity that we consider sufficient for interactive viewing.

Figures & tables

Fig. 1: Training-free conversion of a CT volume rendering into 3D Gaussian splats. The CT volume is shown with a chosen transfer function, which maps Hounsfield units to colour and opacity; the boxes marked Step 1–5 correspond to the steps in the text. A light volume, shared with the photorealistic renderer, stores the visibility of a sky dome in 24 directions and the transmittance towards a key light with soft shadows. The volume is divided into an adaptive octree of millimetre-sized cubic cells, split only where opacity changes, down to sub-voxel size; the finest size is adjusted to reach the requested number of splats. Each cell is sampled at 27 stratified points with the colour and lighting of the volume renderer, giving one colour with baked lighting and an opacity. Opacity mapping replaces each cell by a Gaussian with the same total opacity, fainter at the centre and wider than the cell (

s=0.55h

; top); in a profile through a row of cells, the overlapping Gaussians add up to the opacity of the cells (bottom). Faint splats, whose opacity would fall below the 1/255 floor of standard rasterisers, are retained stochastically with raised opacity. The output is a standard 3DGS file (.ply or .spz); the example shows case 2 with the contrast-enhanced vessel transfer function. CT, computed tomography; 3DGS, three-dimensional Gaussian splatting

Look, views	PSNR (dB)			SSIM	LPIPS	PSNR, real-time
	1 M	2 M	3 M	2 M	2 M	photorealistic (dB)
Bone, wide	32.02 $\pm$ 1.42	32.61 $\pm$ 0.89	32.52 $\pm$ 0.86	0.980 $\pm$ 0.005	0.028 $\pm$ 0.014	44.61 $\pm$ 1.26
Bone, close-up	28.46 $\pm$ 1.87	29.63 $\pm$ 1.55	30.21 $\pm$ 1.42	0.949 $\pm$ 0.012	0.135 $\pm$ 0.041	45.89 $\pm$ 1.64
Vessels, wide	29.17 $\pm$ 1.15	30.09 $\pm$ 0.98	30.43 $\pm$ 0.87	0.960 $\pm$ 0.011	0.036 $\pm$ 0.016	42.28 $\pm$ 1.47
Vessels, close-up	25.23 $\pm$ 1.55	26.56 $\pm$ 1.62	27.29 $\pm$ 1.60	0.895 $\pm$ 0.023	0.150 $\pm$ 0.019	43.89 $\pm$ 1.51
Skin and bone, wide	31.78 $\pm$ 1.80	32.32 $\pm$ 1.51	32.19 $\pm$ 1.18	0.961 $\pm$ 0.017	0.091 $\pm$ 0.040	47.12 $\pm$ 0.79

Table 1: Image quality of the 3DGS models against converged photorealistic references

Fig. 2: Representative results: converged references, 3DGS models and their differences. (a) Bone, case 1; (b) contrast-enhanced vessels, case 6; (c) skin and bone, case 3; one wide view each, cropped to the body. Top: converged photorealistic volume rendering (reference); middle: 3DGS model with two million splats; bottom: difference from the reference (mean absolute difference over the colour channels, 0 to $\geq$ 50 grey levels of 255). Insets (yellow frames) show a close-up view of the same scene; dashed boxes mark the region it covers in the wide view. Errors lie mainly along the edges of dense structures for bone and vessels and are spread over the translucent layer for skin and bone. 3DGS, three-dimensional Gaussian splatting

Look	Views	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6
Bone	wide	33.6/0.982/0.031	32.0/0.971/0.054	33.1/0.980/0.023	31.1/0.979/0.022	33.1/0.980/0.018	32.6/0.985/0.017
Bone	close-up	30.0/0.961/0.178	31.2/0.948/0.192	30.0/0.945/0.124	27.1/0.935/0.118	30.9/0.965/0.085	28.5/0.941/0.114
Vessels	wide	30.2/0.966/0.042	30.4/0.942/0.065	30.3/0.955/0.034	28.3/0.962/0.030	31.3/0.969/0.019	30.1/0.969/0.024
Vessels	close-up	27.4/0.925/0.156	28.7/0.911/0.174	25.9/0.861/0.163	24.0/0.877/0.147	27.4/0.903/0.120	26.1/0.893/0.138
Skin and bone	wide	32.3/0.968/0.088	29.8/0.927/0.170	32.9/0.963/0.087	32.0/0.968/0.070	34.5/0.976/0.064	32.5/0.967/0.069
Skin and bone	close-up	29.4/0.942/0.306	28.7/0.935/0.356	29.9/0.922/0.322	28.2/0.904/0.314	30.5/0.926/0.267	27.9/0.921/0.316

Table 2: Per-case image quality with two million splats, PSNR (dB) / SSIM / LPIPS

Model	1 M	2 M	3 M
Splats (million)	1.04 (0.92–1.37)	2.00 (1.89–2.15)	2.97 (2.76–3.23)
Conversion time, MacBook Pro (s)	11.0 (2.2–24.0)	14.3 (4.2–29.5)	17.1 (3.1–30.7)
Frame rate, MacBook Pro (fps)	122 (97–137)	68 (62–74)	48 (43–53)
Conversion time, Windows desktop (s)	23.1 (4.6–50.0)	29.4 (8.8–60.2)	35.1 (7.4–63.0)
Frame rate, Windows desktop (fps)	52 (36–60)	28 (22–33)	19 (16–22)
Conversion time, MacBook Air 2017 (s)	85 (22–193)	106 (30–216)	— a

Table 3: Splat counts, conversion times and frame rates

Look	Views	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6
Bone	wide	33.6 / 33.4	32.0 / 31.6	33.1 / 32.5	31.2 / 30.8	33.1 / 32.9	32.6 / 32.6
Bone	close-up	30.0 / 29.7	31.2 / 30.7	30.0 / 29.2	27.1 / 26.7	30.9 / 30.0	28.5 / 28.5
Vessels	wide	30.2 / 30.1	30.4 / 29.7	30.3 / 29.9	28.3 / 27.9	31.3 / 30.9	30.1 / 29.7
Vessels	close-up	27.4 / 27.3	28.7 / 27.7	25.9 / 25.4	24.0 / 23.5	27.4 / 26.6	26.1 / 25.6
Skin and bone	wide	32.3 / 29.8	29.8 / 23.8	32.9 / 31.4	32.0 / 31.2	34.5 / 33.7	32.5 / 30.7
Skin and bone	close-up	29.4 / 27.4	28.7 / 23.7	29.9 / 28.6	28.2 / 27.2	30.5 / 29.8	27.9 / 26.5

Table 4: Per-case PSNR (dB) with two million splats, with / without faint splats

Fig. 3: Faint splats: case 2, skin and bone, two million splats. (a) Converged photorealistic volume rendering (reference); (b) 3DGS model without faint splats (ablation of Step 5); (c) 3DGS model with faint splats (CTBakeGS). Top row: one wide view, cropped to the body; middle row: close-up of the dashed box; bottom row: difference from the reference as in Fig. 2 (0 to $\geq$ 50 grey levels of 255). Without them, the thin outer soft-tissue layer, whose splats fall below the 8-bit opacity floor, is missing and the ribs show through (PSNR 22.5 dB in this view); with them it is restored (30.1 dB), with a fine grain visible at close range. 3DGS, three-dimensional Gaussian splatting; PSNR, peak signal-to-noise ratio

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Case	ID	Description	Matrix (voxels)	Voxels (million)	MacBook Pro (s)	Windows desktop (s)	MacBook Air 2017 (s)
1	s0289	Head CT angiography	145 $\times$ 145 $\times$ 233	4.9	5.4	11.1	46
2	s0138	Heart and thoracic aorta	221 $\times$ 221 $\times$ 215	10.5	12.7	25.4	82
3	s1340	Thorax to pelvis	275 $\times$ 275 $\times$ 431	32.6	18.5	37.9	131
4	s0970	Polytrauma	333 $\times$ 333 $\times$ 511	56.7	20.8	43.2	166
5	s1121	Pelvis-to-leg CT angiography	281 $\times$ 281 $\times$ 519	41.0	20.5	42.4	148
6	s0931	Neck-to-leg CT angiography	245 $\times$ 187 $\times$ 479	21.9	8.0	16.6	60

Appendix

Table S1: Cases (TotalSegmentator dataset v2)

Look	Views	Variant	$\Delta$ PSNR (dB)	$p$	$\Delta$ SSIM	$p$	$\Delta$ LPIPS	$p$
Bone	wide	1 M splats	$-0.885$	0.156	$-0.003$	0.312	$+0.005$	0.094
Bone	wide	3 M splats	$+0.036$	1.000	$-0.000$	0.562	$-0.001$	0.438
Bone	wide	No anti-aliasing compensation	$-3.832$	0.031	$-0.010$	0.031	$+0.001$	0.562
Bone	wide	No opacity mapping	$-4.498$	0.031	$-0.012$	0.062	$+0.001$	1.000
Bone	wide	Without faint splats	$-0.274$	0.031	$-0.001$	0.031	$+0.001$	0.031
Bone	wide	Uniform grid (1 cell/voxel)	$-0.171$	0.438	$-0.000$	0.438	$+0.001$	0.219

Appendix

Table S2: Paired comparisons of each variant with the full method (two million splats)

Look	Views	1.5-mm data, millimetre cells	5-mm slices, millimetre cells	5-mm slices, voxel-unit cells
Bone	wide	33.61 / 0.982 / 0.031	34.33 / 0.983 / 0.032	22.54 / 0.898 / 0.109
Bone	close-up	30.03 / 0.961 / 0.178	31.67 / 0.966 / 0.193	17.63 / 0.723 / 0.367
Vessels	wide	30.22 / 0.966 / 0.042	31.02 / 0.964 / 0.043	24.64 / 0.901 / 0.112
Vessels	close-up	27.39 / 0.925 / 0.156	28.00 / 0.924 / 0.181	20.07 / 0.761 / 0.309
Skin and bone	wide	32.26 / 0.968 / 0.088	32.87 / 0.970 / 0.087	20.46 / 0.843 / 0.171
Skin and bone	close-up	29.36 / 0.942 / 0.306	29.77 / 0.950 / 0.289	16.34 / 0.756 / 0.413

Appendix

Table S3: Thick slices: case 1, PSNR (dB) / SSIM / LPIPS

3D Gaussian Splatting

3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.

See all 1,019 papers

Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering
May 22, 2025Zhongpai Gao, Benjamin Planche, Meng Zheng +4Neural Rendering 3D Medical Imaging
United Imaging Intelligence, Boston, MA, USA
TR-GS: High-Fidelity Sparse-View CT Volumetric Rendering via t-Distribution Gaussian Splatting and Ray-Confidence Modeling
Aug 17, 2026Zedong Xiao, Yiren Wang, Zhou Liu +23D Medical Imaging Sparse-View CT Reconstruction
Shenzhen University Shenzhen, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) Shenzhen, China
FaCT-GS: Fast and Scalable CT Reconstruction with Gaussian Splatting
Apr 2, 2026Pawel Tomasz Pieta, Rasmus Juul Pedersen, Sina Borgi +3Computed Tomography 3DGS Acceleration
Technical University of Denmark, Kgs. Lyngby, Denmark

May 22, 2025J/K move · Enter open · S save

Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering

Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri+3

United Imaging Intelligence, Boston, MA, USA

Neural Rendering 3D Medical Imaging Gaussian Splatting Computed Tomography

Open paper AskPDF arXiv

Photorealistic volumetric rendering of CT scans greatly benefits clinical workflows, yet neural approaches such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) require prohibitive per-scan optimization (hours for NeRF, about 30 minutes for 3DGS), making them impractical in clinical settings. We propose Render-FM, a feedforward model that eliminates this bottleneck by directly regressing 6D Gaussian Splatting (6DGS) parameters from a CT volume in a single 2.8-second forward pass, a 500x speedup over per-scan optimization. To bridge the domain gap between natural scene reconstruction and medical volumetric rendering, we introduce Anatomy-Guided Priming (AGP), which incorporates segmentation masks and transfer functions as structural and appearance priors, information that existing Gaussian splatting methods overlook. Built on an nnU-Net-inspired 3D U-Net trained on diverse CT scans, Render-FM predicts per-voxel 6DGS parameters and supports immediate real-time rendering. Unlike per-scan methods, it generalizes to unseen anatomies, novel transfer functions, and enables compositional organ visualization with zero additional preparation time. Optional 89-second fine-tuning further improves quality, surpassing per-scan optimized baselines. Project page: https://gaozhongpai.github.io/renderfm/.

Abstract

3D Gaussian Splatting

Explore similar work

Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering

TR-GS: High-Fidelity Sparse-View CT Volumetric Rendering via t-Distribution Gaussian Splatting and Ray-Confidence Modeling

FaCT-GS: Fast and Scalable CT Reconstruction with Gaussian Splatting