Temporal Residual Neural Radiance Fields for Monocular Video Dynamic Human Body Reconstruction

Tianle Du; Jie Wang; Xiaolong Xie; Wei Li; Pengxiang Su; Jie Liu

cs.CVSep 4, 2026

Temporal Residual Neural Radiance Fields for Monocular Video Dynamic Human Body Reconstruction

Authors: Tianle Du, Jie Wang, Xiaolong Xie, Wei Li, Pengxiang Su, Jie Liu

Organizations: Nanchang University

Abstract

In the field of computer vision and graphics, high-quality reconstruction of the human body in static scenes has been achieved in recent years by a single multilayer perceptron (MLP) in a number of approaches. However, MLPs have capacity limitations, requiring substantial training time and computational resources for dynamic scene reconstruction. And the quality of reconstruction is significantly constrained. This paper proposes a method for effectively processing complex spatiotemporal signals in dynamic scene human 3D modeling. The proposed method uses Temporal Residual Neural Radiance Fields to achieve novel view rendering and new pose synthesis of human bodies.To address the problem of representing temporal signals in video sequences, we construct a temporal residual field which is not related to the MLP architecture. Secondly, to improve reconstruction efficiency, we propose an integrated approach that reduces trainable parameters and accelerates rendering, thereby enhancing the network's feature representation capability. Finally, we design a multi-dimensional loss function to accurately measure the loss between predicted and actual spatial pixel values. The experimental results show that our proposed approach improves the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) accuracy metrics compared to the latest representative methods. It maintains similar accuracy to Anim-NeRF and Neural Body while achieving a nearly 780-fold increase in time efficiency.

Figures & tables

Figure 1: Overview of TRes-NeRF Network Structure. For a given sequence of human videos, after applying rigid transformations to the human joints, we utilize an occupancy grid to filter out points in empty space to reduce the computational load. The remaining points are then extracted by the TRes-NeRF network to extract time-dependent features among the image sequences and are input into a multiresolution hash coding structure along with spatial location features to evaluate their color and density.

	Mean Trainin $\downarrow$	Mean Rendering $\uparrow$
w/o empty voxel skipping	3m16s	7 FPS
w/ empty voxel skipping	1m38s	red 13 FPS

Table 1: Empty voxel skipping

	male-3-casual		male-4-casual		female-3-casual		female-4-casual
	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
Neural Body Peng et al. (2021b) (14 hours)	24.94	0.9428	24.71	0.9496	23.87	0.9504	24.37	0.9451
Anim-NeRF Chen et al. (2021) (13 hours)	29.37	0.9703	28.37	0.9605	28.91	0.9743	28.90	0.9678
InstantAvatar Jiang et al. (2023) (1 minute)	29.65	0.9730	27.97	0.9649	27.90	0.9722	28.92	0.9692
Ours (1 minute)	29.89	0.9735	28.12	0.9652	28.42	0.9728	29.56	0.9714

Table 2: Qualitative comparison with SoTA methods on the PeopleSnapshot[30] dataset.

Figure 2: Results of human body reconstruction on the PeopleSnapshot dataset Liu et al. (2020) (above) and the Neuman dataset Jiang et al. (2023) (below) demonstrate that TRes-NeRF excels in reconstructing details.

	bike		citron		jogging		lab		parkinglot		seattle
	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
InstantAvatar Jiang et al. (2023)	24.18	0.9498	24.95	0.9491	23.40	0.9338	27.34	0.9731	24.14	0.9520	26.57	0.9674
Ours	24.32	0.9487	24.99	0.9496	23.49	0.9339	27.54	0.9697	24.45	0.9528	26.66	0.9659

Table 3: Qualitative results on the Neuman dataset. Jiang et al. (2023)

Figure 3: More challenging human body reconstruction results on the PeopleSnapshot Alldieck et al. (2018) dataset.The reconstruction results of TRes-NeRF and InstantAvatar on challenging human models in the PeopleSnapshot dataset. The images demonstrate that TRes-NeRF is capable of effectively eliminating artifacts in the reconstruction.

	male-3-sport		male-2-sport		male-2-casual		female-4-sport
	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
InstantAvatar Jiang et al. (2023) (1 minute)	23.67	0.9216	24.99	0.9277	25.25	0.9306	22.04	0.8930
Ours(1 minute)	23.98	0.9281	24.92	0.9304	25.15	0.9270	22.30	0.9021

Table 4: Qualitative comparison of more challenging data in the PeopleSnapshot dataset. Liu et al. (2020)

Figure 4: More human body reconstruction results of TRes-NeRF on the PeopleSnapshot dataset Liu et al. (2020) , including the synthesis effects of new poses.

	Mean PSNR $\uparrow$	Mean SSIM $\uparrow$	GPU $\downarrow$
w/o Time Residual	25.24	0.9540	3.8G
w/ Time Residual	25.62	0.9586	2.4G

Table 5: Time residual network

	Mean Trainin $\downarrow$	Mean Rendering $\uparrow$
w/o TRes Factorization	4m24s	1 FPS
w/ TRes Factorization	1m36s	13 FPS

Table 6: TRes-NeRF Factor Decomposition

3D Human Reconstruction

3D Human Reconstruction recovers human shape, pose, appearance, and motion from images, video, or other sensor data.

See all 105 papers

Few TensoRF: Enhance the Few-shot on Tensorial Radiance Fields
Mar 26, 2026Thanh-Hai Le, Hoang-Hau Tran, Trong-Nghia VuNeural Radiance Fields Sparse-View 3D Reconstruction
School of Computer Science and Engineering, The Saigon International University, Ho Chi Minh City 700000, Vietnam · Department of Informatic Technology, FPT University, Ho Chi Minh City 700000, Vietnam
GUSH3R: Everyone Everywhere All at Once as Gaussians
Jul 6, 2026Keito Abe, Kaede Shiohara, Takashi Otonari +13D Gaussian Splatting 3D Reconstruction
The University of Tokyo
FlexiAvatar: Unified 3D Gaussian Human Avatars Under Arbitrary Body Visibility
Jul 21, 2026Yihalem Yimolal Tiruneh, Muhammad Salman Ali, Uyoung Jeong +53D Gaussian Splatting 3D Reconstruction
UNIST, South Korea · University of Aberdeen, UK · University College London, UK +1

Mar 26, 2026J/K move · Enter open · S save

Few TensoRF: Enhance the Few-shot on Tensorial Radiance Fields

Thanh-Hai Le, Hoang-Hau Tran, Trong-Nghia Vu

School of Computer Science and Engineering, The Saigon International University, Ho Chi Minh City 700000, Vietnam · Department of Informatic Technology, FPT University, Ho Chi Minh City 700000, Vietnam

Neural Radiance Fields Sparse-View 3D Reconstruction 3D Human Reconstruction

Open paper AskPDF arXiv

This paper presents Few TensoRF, a 3D reconstruction framework that combines TensorRF's efficient tensor based representation with FreeNeRF's frequency driven few shot regularization. Using TensorRF to significantly accelerate rendering speed and introducing frequency and occlusion masks, the method improves stability and reconstruction quality under sparse input views. Experiments on the Synthesis NeRF benchmark show that Few TensoRF method improves the average PSNR from 21.45 dB (TensorRF) to 23.70 dB, with the fine tuned version reaching 24.52 dB, while maintaining TensorRF's fast

\approx10-15

minute training time. Experiments on the THuman 2.0 dataset further demonstrate competitive performance in human body reconstruction, achieving 27.37 - 34.00 dB with only eight input images. These results highlight Few TensoRF as an efficient and data effective solution for real-time 3D reconstruction across diverse scenes.

Abstract

3D Human Reconstruction

Explore similar work

Few TensoRF: Enhance the Few-shot on Tensorial Radiance Fields

GUSH3R: Everyone Everywhere All at Once as Gaussians

FlexiAvatar: Unified 3D Gaussian Human Avatars Under Arbitrary Body Visibility