Endo-TSR: Temporal Spectral Modeling of Appearance and Motion for Endoscopic Reconstruction

Taoyu Wu; Yiyi Miao; Qi Shao; Zhuoxiao Li; Zhe Tang; Limin Yu; Baoru Huang

cs.CVSep 26, 2026

Endo-TSR: Temporal Spectral Modeling of Appearance and Motion for Endoscopic Reconstruction

Authors: Taoyu Wu, Yiyi Miao, Qi Shao, Zhuoxiao Li, Zhe Tang, Limin Yu, Baoru Huang

Organizations: Academy of Artificial Intelligence and Advanced Technology, Xi’an Jiaotong-Liverpool University, China · School of Computer Science and Informatics, University of Liverpool, United Kingdom · Hong Kong University of Science and Technology (Guangzhou), China · Institute of Artificial Intelligence Innovation, Zhejiang University of Technology, China

Abstract

Endoscopic scene reconstruction requires modeling tissue motion and temporal appearance while recovering fine surface detail. Deformable Gaussian models provide explicit trajectories, but their fixed colour coefficients lack a dedicated temporal representation for photometric changes. We propose Endo-TSR, which augments deformable Gaussian splatting with bounded Fourier colour residuals and independent translation residuals on shared temporal frequencies. The colour residuals capture local appearance changes, while a Matérn spectral prior regularises motion corrections. Multi-scale Laplacian supervision guides tissue-detail recovery during joint image fitting. Extensive experiments on the EndoNeRF and StereoMIS datasets demonstrate state-of-the-art rendering quality, with the highest PSNR across all evaluated sequences. Ablation studies show that temporal appearance yields the largest PSNR gain among the tested component additions, while appearance and detail supervision jointly improve rendering with fixed Gaussian counts.

Figures & tables

Figure 1: Overview of Endo-TSR. RGB images and depth priors establish base Gaussians, whose persistent colour and FDM trajectories form the basis of the time-varying representation. In parallel, a shared Fourier dictionary and independent coefficients produce bounded colour residuals and translation corrections. These residuals augment the base quantities before rasterisation at the input camera pose. Training combines base reconstruction losses, multi-scale Laplacian supervision over valid tissue regions, and a Matérn penalty on motion coefficients. Dashed Gaussian outlines indicate previous states. Solid arrows denote forward flow. Black dashed arrows return supervision to the residual coefficients, while purple dashed arrows connect the motion prior to the translation branch and total objective.

	EndoNeRF–Cutting			EndoNeRF–Pulling			StereoMIS–P1			StereoMIS–P2-2			StereoMIS–P2-5			StereoMIS–P3
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
EndoNeRF [ 2 ]	35.71	0.933	0.059	36.50	0.941	0.054	31.12	0.846	0.148	28.91	0.703	0.261	26.49	0.688	0.274	29.85	0.751	0.209
EndoSurf [ 4 ]	36.55	0.953	0.055	37.21	0.955	0.050	31.29	0.865	0.168	30.01	0.780	0.198	26.51	0.738	0.277	30.46	0.811	0.188
ForPlane-32k [ 5 ]	36.85	0.944	0.045	37.82	0.955	0.042	32.19	0.843	0.143	28.01	0.659	0.286	28.48	0.759	0.170	31.10	0.801	0.139
Endo-4DGS [ 9 ]	36.56	0.955	0.032	37.85	0.959	0.043	32.17	0.850	0.192	31.45	0.812	0.141	29.06	0.817	0.152	31.21	0.831	0.155
EndoGaussian [ 8 ]	38.38	0.963	0.030	37.09	0.956	0.039	32.79	0.858	0.182	29.82	0.759	0.169	28.28	0.782	0.173	31.26	0.821	0.166

Table 1: Rendering quality on EndoNeRF and StereoMIS under the Endo-PairGS benchmark protocol [ 12 ] , with full-image averaging. Our results are averaged over five seeds on EndoNeRF and three on StereoMIS. Bold: best per column.

			EndoNeRF			StereoMIS
A	S	L	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
$\times$	$\times$	$\times$	38.08	0.9537	0.0309	34.40	0.8959	0.0699
$\times$	$\checkmark$	$\times$	38.12	0.9540	0.0302	34.44	0.8964	0.0686
$\checkmark$	$\checkmark$	$\times$	38.40	0.9563	0.0280	34.74	0.9005	0.0643
$\checkmark$	$\checkmark$	$\checkmark$	38.52	0.9569	0.0271	34.81	0.9015	0.0625

Table 2: Component ablation. A: temporal appearance. S: spectral motion refinement. L: multi-scale detail supervision. Scores weight sequence means equally. PSNR/SSIM use tissue pixels, while LPIPS uses masked images. Seeds: five for EndoNeRF (base: three), three for StereoMIS. Best results are bold.

Endoscopic 3D Reconstruction

See all 18 papers

EndoPrior-GS: Dynamic Endoscopic Reconstruction with a Joint Texture Prior
Sep 29, 2026Jiaqi Huang, Shidong Wang, Tong Xin +13D Gaussian Splatting Medical Imaging
School of Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, UK · School of Computing, Newcastle University, Newcastle upon Tyne NE4 5TG, UK
4D Monocular Surgical Reconstruction under Arbitrary Camera Motions
Feb 19, 2026Jiwei Shan, Zeyu Cai, Cheng-Tai Hsieh +54D Reconstruction 3D Reconstruction
Department of Mechanical and Automation Engineering and T Stone Robotics Institute, The Chinese University of Hong Kong, Hong Kong. · Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China · State Key Laboratory of Robotics and Intelligent Systems, Shenyang, China +3
Endo-NeRF++: Uncertainty-Aware Neural Rendering with Multi-Resolution Hash Encoding for Dynamic Surgical Scene Reconstruction
Jul 30, 2026Gousia Habib, Laura Ruotsalainen3D Reconstruction Neural Rendering
Department of Computer Science, University of Helsinki, Finland

Sep 29, 2026J/K move · Enter open · S save

EndoPrior-GS: Dynamic Endoscopic Reconstruction with a Joint Texture Prior

Jiaqi Huang, Shidong Wang, Tong Xin, Kabita Adhikari

School of Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, UK · School of Computing, Newcastle University, Newcastle upon Tyne NE4 5TG, UK

3D Gaussian Splatting Medical Imaging Dynamic Scene Reconstruction Deformable Shape Reconstruction Endoscopic 3D Reconstruction

Open paper AskPDF arXiv

Dynamic endoscopic reconstruction is fundamental to robotic surgery and computer-assisted interventions. While 3D Gaussian Splatting (3DGS) realises real-time rendering, its application to deformable intraoperative environments remains constrained by spurious geometry and varying illuminations. To address these limitations, we introduce EndoPrior-GS, a novel pipeline that explicitly couples frame-extracted vision heuristics and estimated depth maps. EndoPrior-GS derives a joint texture prior from a tool-filtered valid tissue mask, a non-specular photometric filter, and anatomical structural salience, yielding a probability map that guides primitive initialisation and subsequent density control. The prior is further extended to the temporal domain through a texture-aware term that dynamically weighs pairwise primitive contributions during training. We conduct extensive experiments on benchmark datasets EndoNeRF and SCARED, and the obtained results show that our method EndoPrior-GS reduces Flow Error by 27.7% and 25.8% over the representative approaches while preserving competitive rendering quality and real-time rendering speed. Our project website is available at https://jiaqi-huang-77.github.io/EndoPrior-GS/.

Abstract

Endoscopic 3D Reconstruction

Explore similar work

EndoPrior-GS: Dynamic Endoscopic Reconstruction with a Joint Texture Prior

4D Monocular Surgical Reconstruction under Arbitrary Camera Motions

Endo-NeRF++: Uncertainty-Aware Neural Rendering with Multi-Resolution Hash Encoding for Dynamic Surgical Scene Reconstruction