Shared-Structure 4D Spectral Gaussian Representation for Sparse-View Spectral CT Reconstruction

Jiancheng Fang; Shaoyu Wang; Wenjun Xia; Yang Chen; Qiegen Liu

cs.CVAug 17, 2026

Shared-Structure 4D Spectral Gaussian Representation for Sparse-View Spectral CT Reconstruction

Authors: Jiancheng Fang, Shaoyu Wang, Wenjun Xia, Yang Chen, Qiegen Liu

Organizations: School of Information Engineering, Nanchang University, Nanchang 330031, China · School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, Laboratory of Image Science and Technology, and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education

Abstract

Sparse-view spectral computed tomography (CT) reconstructs energy-resolved attenuation volumes from limited projection views, requiring simultaneous handling of angular undersampling and spectral coupling. We propose a SharedStructure 4D Spectral Gaussian Representation (4D-SG) that learns shared Gaussian geometry from full spectrum structural projections and uses a Gaussian-wise Spectral Density Curve Network (GSC-Net) to predict Gaussian raw density transformations. This factorization separates shared spatial structure from spectral attenuation variation, avoids independent channel geometry optimization, and establishes a continuous 4D-SG representation from discrete spectral measurements for unobserved spectral channel queries. Experiments on six synthesized, simulated projection, and real projection datasets with 50 views demonstrate the best average performance. Compared with the strongest Gaussian baseline, 4D-SG improves PSNR from 35.56 dB to 36.61 dB, increases SSIM from 0.909 to 0.914, and reduces LPIPS from 0.208 to 0.194, demonstrating its effectiveness for sparse-view spectral CT reconstruction.

Figures & tables

Fig. 1: Conceptual comparison. (a) Conventional methods reconstruct each spectral channel through an independent iterative process. (b) 4D-SG learns shared Gaussian geometry and GSC-Net from discrete spectral measurements and sparse-view projections and generates the Gaussian representation of a target channel for voxelization, supporting reconstruction and prediction of observed and unobserved spectral channels.

Fig. 2: Empirical analysis of structural consistency across channels and attenuation variation by region in spectral CT. (a) Representative spectral CT slices, corresponding edge maps, and distributions of the shared structural edge ratio under different thresholds $k$ for the minimum number of concurring channels. Violin plots, samples from individual slices, box plots, and median markers summarize the distributions; the dashed line denotes 0.95, and the bottom annotations report the proportion of samples with low ratios and the minimum ratio. (b) Four fixed regions of interest, enlarged patches, and their mean globally normalized attenuation values across spectral channels, with the measured values annotated at each channel.

Fig. 3: Workflow of 4D-SG. Shared Gaussian geometry and common raw densities are first learned from structural projections over the full spectrum using projection fidelity and volumetric regularization. With the Gaussian centers, scales, and rotations fixed, GSC-Net predicts raw density transformations for each Gaussian conditioned on the common raw densities, spectral grid, and target channel. The resulting density coefficients for the target channel are combined with the fixed shared geometry and optimized using spectral projection fidelity, curve regularization, and volumetric regularization. Gaussian voxelization reconstructs the queried spectral channel, supporting reconstruction of observed spectral channels and queries of unobserved spectral channels.

Fig. 4: Architecture of GSC-Net. The common raw densities and normalized spectral grid are embedded into spectral token sequences for each Gaussian and processed by the gated spectral SSM backbone. The density curve head combines token, low-rank basis, and residual responses to predict raw density transformations for each Gaussian. Observed spectral channels are queried directly from the predicted curves, whereas target spectral channels are queried from the predicted raw density curves before conversion to non-negative density coefficients.

Dataset	Data Source	Spectral Setting	SOD/SDD(mm)	Detector	Reference Volume
Chest [ 32 ]	Synthesized from a single-energy CT volume	3 monoenergetic channels: 40, 70, and 100 keV	1000/1500	$512\times 512$ pixels, 1-mm pixel size	Synthesized GT
Head [ 33 ]	Synthesized from a single-energy CT volume	3 monoenergetic channels: 40, 70, and 100 keV	1000/1500	$512\times 512$ pixels, 1-mm pixel size	Synthesized GT
Walnut [ 34 ]	Simulated from a spectral CT volume	8 monoenergetic channels: 10–80 keV at 10-keV intervals	1000/1500	$512\times 512$ pixels, 1-mm pixel size	Spectral CT volume
Ordinary Objects [ 35 ]	Simulated from dual-energy CT volumes	5 mixed-energy channels from 100/140-kVp volumes, with $\lambda=\{0,0.3,0.5,0.7,1.0\}$	1000/1500	$512\times 512$ pixels, 1-mm pixel size	Spectral CT volume
Insect	Real multi-kVp cone-beam CT projections	4 tube-voltage settings: 40, 60, 80, and 100 kVp	310/510	$2882\times 2340$ pixels, 0.05-mm pixel size	Full-view FDK
Mouse Chest	Real photon-counting CT projections	4 energy windows: 20–90, 30–90, 45–90, and 60–90 keV	76.28/361.1	$2062\times 2062$ pixels, 0.1-mm pixel size	Full-view FDK

TABLE I: Summary of the datasets, spectral configurations, and acquisition geometries. Chest, Head, Walnut, and Ordinary Objects use the same simulated cone-beam system matrix, while full-view FDK reconstructions are used as reference volumes for Insect and Mouse Chest.

Method	Chest			Head			Walnut			Ordinary Objects			Insect			Mouse Chest			Average
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
FDK	27.00	0.546	0.402	29.41	0.702	0.346	24.77	0.540	0.393	29.85	0.625	0.326	27.09	0.526	0.440	22.61	0.516	0.432	26.79	0.576	0.390
CGLS	33.43	0.858	0.244	35.08	0.909	0.256	29.99	0.813	0.283	37.74	0.829	0.243	30.50	0.877	0.239	27.43	0.669	0.419	32.36	0.826	0.281
SART	32.85	0.886	0.228	36.73	0.925	0.256	32.76	0.897	0.244	39.26	0.943	0.225	31.96	0.900	0.236	28.78	0.705	0.361	33.72	0.876	0.258
NAF	35.56	0.896	0.192	37.35	0.942	0.215	33.68	0.939	0.199	40.12	0.968	0.193	32.74	0.914	0.194	28.80	0.715	0.349	34.71	0.896	0.224
R 2 -Gaussian	35.77	0.911	0.174	38.15	0.947	0.209	34.93	0.948	0.186	40.34	0.974	0.170	35.06	0.924	0.146	29.08	0.751	0.360	35.56	0.909	0.208

TABLE II: Quantitative comparison on all datasets using 50 views. Higher PSNR and SSIM values are better, whereas lower LPIPS values are better. Metrics on Insect and Mouse Chest are computed with respect to full-view FDK references.

Fig. 5: Representative qualitative comparison on 3D volume datasets with simulated projections using 50 views. The upper two groups show Walnut at 30 keV and 70 keV, respectively, and the lower two groups show Ordinary Objects with mixing coefficients of 0.0 and 1.0, respectively. For each spectral setting, the first row shows the reconstructed slice and the second row shows the corresponding zoomed ROI. The red boxes indicate the enlarged regions, and the red arrows highlight representative local structures. The PSNR and SSIM values annotated in each panel are computed on the corresponding entire 3D volume at the displayed spectral channel, rather than on the shown slice only. HU values are normalized to [0,1].

Fig. 6: Representative qualitative comparison on synthesized multi-energy datasets using 50 views. The upper two groups show Chest at 40 keV and 100 keV, respectively, and the lower two groups show Head at 40 keV and 100 keV, respectively. For each spectral setting, the first row shows the reconstructed slice and the second row shows the corresponding zoomed ROI. The red boxes indicate the enlarged regions, and the red arrows highlight representative local structures. The PSNR and SSIM values annotated in each panel are computed on the corresponding entire 3D volume at the displayed spectral channel, rather than on the shown slice only. HU values are normalized to [0,1].

Fig. 7: Representative qualitative comparison on real projection datasets using 50 views. The upper two groups show Insect acquired at 40 kVp and 100 kVp, respectively, and the lower two groups show Mouse Chest at the 20–90 keV and 60–90 keV energy windows, respectively. For each spectral setting, the first row shows the reconstructed slice and the second row shows the corresponding zoomed ROI. The red boxes indicate the enlarged regions, and the red arrows highlight representative local structures. Since clean ground-truth volumes are unavailable for the real projection datasets, the reference volumes are full-view FDK reconstructions. The PSNR and SSIM values annotated in each panel are computed on the corresponding entire 3D volume at the displayed spectral channel, rather than on the shown slice only.

Fig. 8: Reconstruction and quantitative evaluation across spectral channels. (a) Reconstructed and reference Walnut slices. (b) PSNR, SSIM, and LPIPS by channel for each dataset.

Fig. 9: Ablation on Gaussian parameter freezing during raw density curve learning. Rows report PSNR, SSIM, and LPIPS. P/R/S/D denote position, rotation, scale, and raw density; L/F denote learnable/frozen.

Fig. 10: Comparison between GSC-Net and a direct MLP alternative under different numbers of projection views. Higher values are better for PSNR and SSIM, whereas lower values are better for LPIPS.

Fig. 11: Comparison of training time and reconstruction quality for different spectral channel counts and configurations. The top plot reports how total training time changes as the number of spectral channels increases. It compares independent R 2 -Gaussian training against shared geometry variants using either MLP or GSC-Net for spectral adaptation. The bottom plots report PSNR, SSIM, and LPIPS. These results indicate that reusing shared Gaussian geometry improves training efficiency while maintaining reconstruction accuracy.

Fig. 12: Prediction of projections for an unobserved spectral channel across six datasets. (a) Predicted projections, corresponding reference projections, and residual maps. (b) Projection-level metrics evaluated across multiple views.

Explore similar work

SpectralCTGaussians: Projection-Domain Reconstruction and Basis Material Decomposition for Spectral CT using 3D Gaussian Splatting
Aug 29, 2026Reinout Vos, Saptarshi Neil Sinha, Michael WeinmannComputed Tomography 3D Gaussian Splatting
Delft University of Technology, Delft, Netherlands · Fraunhofer IGD, Darmstadt, Germany
CG-GLORE: A Conjugate Gradient-Based Global-Local Regularization Network for Sparse-View CT Reconstruction
Aug 15, 2026Tran Xuan Hieu Le, Doanh C. Bui, Vu Trung Duong Le +5Sparse-View CT Reconstruction Computed Tomography
Nara Institute of Science and Technology Nara, Japan · University of Information Technology Ho Chi Minh City, Vietnam · Vietnam National University Ho Chi Minh City, Vietnam +2
SONAR: A Structure-Consistent Neural Operator for Null-Space-Aware Sparse View CT Reconstruction
Sep 12, 2026Song Ni, Haijun Yu, Haodong Li +4Sparse-View CT Reconstruction Image Inverse Problems
Department of Electrical and Computer Engineering, University of Massachusetts Lowell, Lowell, MA 01854

Aug 29, 2026J/K move · Enter open · S save

SpectralCTGaussians: Projection-Domain Reconstruction and Basis Material Decomposition for Spectral CT using 3D Gaussian Splatting

Reinout Vos, Saptarshi Neil Sinha, Michael Weinmann

Delft University of Technology, Delft, Netherlands · Fraunhofer IGD, Darmstadt, Germany

Computed Tomography 3D Gaussian Splatting

Open paper AskPDF arXiv

Spectral computed tomography (CT) extends conventional CT by measuring attenuation across multiple energy channels, allowing improved modeling of physical X-ray interactions and energy-dependent material behavior and leading to richer scene understanding. We present a novel method for spectral CT reconstruction and basis material decomposition using 3D Gaussian Splatting by adding per-Gaussian basis material fractions to the set of learnable parameters, which together with a set of energy-dependent basis functions define the attenuation across the full spectral range. The basis functions represent various physical attenuation models such as photoelectric absorption and Compton scattering, and are jointly optimized across all energy channels through a differentiable polychromatic forward model, with material decomposition performed via mean-shift clustering of the resulting coefficients. We evaluate our method on a baseline real-world dataset as well as a synthetic dataset that we introduce, comparing against traditional reconstruction algorithms and state-of-the-art learning-based CT reconstruction methods. Our approach outperforms all traditional baselines in novel view synthesis and achieves the best PSNR among all compared methods for spectral CT volume reconstruction, while describing all energy channels with a single shared representation that requires a number of Gaussians comparable to single-channel Gaussian splatting-based CT reconstruction approaches. For basis material decomposition, no traditional or learning-based baseline offers one-step decomposition with direct RGB material segmentation, and our method additionally recovers the photoelectric basis with higher PSNR than traditional pipelines.

Abstract

Explore similar work

SpectralCTGaussians: Projection-Domain Reconstruction and Basis Material Decomposition for Spectral CT using 3D Gaussian Splatting

CG-GLORE: A Conjugate Gradient-Based Global-Local Regularization Network for Sparse-View CT Reconstruction

SONAR: A Structure-Consistent Neural Operator for Null-Space-Aware Sparse View CT Reconstruction