Temporal Residual Neural Radiance Fields for Monocular Video Dynamic Human Body Reconstruction

Tianle Du; Jie Wang; Xiaolong Xie; Wei Li; Pengxiang Su; Jie Liu

cs.CVSep 4, 2026

Temporal Residual Neural Radiance Fields for Monocular Video Dynamic Human Body Reconstruction

Authors: Tianle Du, Jie Wang, Xiaolong Xie, Wei Li, Pengxiang Su, Jie Liu

Organizations: Nanchang University

Abstract

In the field of computer vision and graphics, high-quality reconstruction of the human body in static scenes has been achieved in recent years by a single multilayer perceptron (MLP) in a number of approaches. However, MLPs have capacity limitations, requiring substantial training time and computational resources for dynamic scene reconstruction. And the quality of reconstruction is significantly constrained. This paper proposes a method for effectively processing complex spatiotemporal signals in dynamic scene human 3D modeling. The proposed method uses Temporal Residual Neural Radiance Fields to achieve novel view rendering and new pose synthesis of human bodies.To address the problem of representing temporal signals in video sequences, we construct a temporal residual field which is not related to the MLP architecture. Secondly, to improve reconstruction efficiency, we propose an integrated approach that reduces trainable parameters and accelerates rendering, thereby enhancing the network's feature representation capability. Finally, we design a multi-dimensional loss function to accurately measure the loss between predicted and actual spatial pixel values. The experimental results show that our proposed approach improves the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) accuracy metrics compared to the latest representative methods. It maintains similar accuracy to Anim-NeRF and Neural Body while achieving a nearly 780-fold increase in time efficiency.

Figures & tables

Figure 1: Overview of TRes-NeRF Network Structure. For a given sequence of human videos, after applying rigid transformations to the human joints, we utilize an occupancy grid to filter out points in empty space to reduce the computational load. The remaining points are then extracted by the TRes-NeRF network to extract time-dependent features among the image sequences and are input into a multiresolution hash coding structure along with spatial location features to evaluate their color and density.

	Mean Trainin $\downarrow$	Mean Rendering $\uparrow$
w/o empty voxel skipping	3m16s	7 FPS
w/ empty voxel skipping	1m38s	red 13 FPS

Table 1: Empty voxel skipping

	male-3-casual		male-4-casual		female-3-casual		female-4-casual
	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
Neural Body Peng et al. (2021b) (14 hours)	24.94	0.9428	24.71	0.9496	23.87	0.9504	24.37	0.9451
Anim-NeRF Chen et al. (2021) (13 hours)	29.37	0.9703	28.37	0.9605	28.91	0.9743	28.90	0.9678
InstantAvatar Jiang et al. (2023) (1 minute)	29.65	0.9730	27.97	0.9649	27.90	0.9722	28.92	0.9692
Ours (1 minute)	29.89	0.9735	28.12	0.9652	28.42	0.9728	29.56	0.9714

Table 2: Qualitative comparison with SoTA methods on the PeopleSnapshot[30] dataset.

Figure 2: Results of human body reconstruction on the PeopleSnapshot dataset Liu et al. (2020) (above) and the Neuman dataset Jiang et al. (2023) (below) demonstrate that TRes-NeRF excels in reconstructing details.

	bike		citron		jogging		lab		parkinglot		seattle
	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
InstantAvatar Jiang et al. (2023)	24.18	0.9498	24.95	0.9491	23.40	0.9338	27.34	0.9731	24.14	0.9520	26.57	0.9674
Ours	24.32	0.9487	24.99	0.9496	23.49	0.9339	27.54	0.9697	24.45	0.9528	26.66	0.9659

Table 3: Qualitative results on the Neuman dataset. Jiang et al. (2023)

Figure 3: More challenging human body reconstruction results on the PeopleSnapshot Alldieck et al. (2018) dataset.The reconstruction results of TRes-NeRF and InstantAvatar on challenging human models in the PeopleSnapshot dataset. The images demonstrate that TRes-NeRF is capable of effectively eliminating artifacts in the reconstruction.

	male-3-sport		male-2-sport		male-2-casual		female-4-sport
	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
InstantAvatar Jiang et al. (2023) (1 minute)	23.67	0.9216	24.99	0.9277	25.25	0.9306	22.04	0.8930
Ours(1 minute)	23.98	0.9281	24.92	0.9304	25.15	0.9270	22.30	0.9021

Table 4: Qualitative comparison of more challenging data in the PeopleSnapshot dataset. Liu et al. (2020)

Figure 4: More human body reconstruction results of TRes-NeRF on the PeopleSnapshot dataset Liu et al. (2020) , including the synthesis effects of new poses.

	Mean PSNR $\uparrow$	Mean SSIM $\uparrow$	GPU $\downarrow$
w/o Time Residual	25.24	0.9540	3.8G
w/ Time Residual	25.62	0.9586	2.4G

Table 5: Time residual network

	Mean Trainin $\downarrow$	Mean Rendering $\uparrow$
w/o TRes Factorization	4m24s	1 FPS
w/ TRes Factorization	1m36s	13 FPS

Table 6: TRes-NeRF Factor Decomposition

3D Human Reconstruction

3D Human Reconstruction recovers human shape, pose, appearance, and motion from images, video, or other sensor data.

See all 105 papers

Mar 26, 2026

Few TensoRF: Enhance the Few-shot on Tensorial Radiance Fields

This paper presents Few TensoRF, a 3D reconstruction framework that combines TensorRF's efficient tensor based representation with FreeNeRF's frequency driven few shot regularization. Using TensorRF to significantly accelerate rendering speed and introducing frequency and occlusion masks, the method improves stability and reconstruction quality under sparse input views. Experiments on the Synthesis NeRF benchmark show that Few TensoRF method improves the average PSNR from 21.45 dB (TensorRF) to 23.70 dB, with the fine tuned version reaching 24.52 dB, while maintaining TensorRF's fast

\approx10-15

minute training time. Experiments on the THuman 2.0 dataset further demonstrate competitive performance in human body reconstruction, achieving 27.37 - 34.00 dB with only eight input images. These results highlight Few TensoRF as an efficient and data effective solution for real-time 3D reconstruction across diverse scenes.

Thanh-Hai Le, Hoang-Hau Tran, Trong-Nghia Vu

School of Computer Science and Engineering, The Saigon International University, Ho Chi Minh City 700000, Vietnam · Department of Informatic Technology, FPT University, Ho Chi Minh City 700000, Vietnam

Neural Radiance Fields Sparse-View 3D Reconstruction 3D Human Reconstruction

Ask PDF

Jul 6, 2026

GUSH3R: Everyone Everywhere All at Once as Gaussians

Reconstructing dynamic human-scene environments from monocular videos is a challenging problem that requires jointly modeling scene geometry, camera motion, and non-rigid human dynamics while enabling photorealistic rendering. Recent feed-forward methods can efficiently predict geometry, but they are often limited to non-photorealistic representations such as point clouds and meshes, or they fail to handle non-rigid objects, particularly dynamic humans. To fill this gap, we present GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction), a feed-forward framework for online dynamic human-scene reconstruction. From a monocular human-scene video, our method reconstructs dynamic humans (everyone) and static scenes (everywhere) in a single forward pass (all at once) as 3D Gaussian Splatting (3DGS) primitives (as gaussians), which are geometrically consistent and capable of novel view synthesis. Experiments on monocular human-scene datasets demonstrate that our approach achieves competitive novel view synthesis quality while significantly improving inference efficiency compared to optimization-based methods.

Keito Abe, Kaede Shiohara, Takashi Otonari +1

The University of Tokyo

3D Gaussian Splatting 3D Reconstruction Monocular 3D Reconstruction Dynamic Scene Reconstruction Novel View Synthesis 3D Human Reconstruction+1

Ask PDF

Jul 21, 2026

FlexiAvatar: Unified 3D Gaussian Human Avatars Under Arbitrary Body Visibility

Reconstructing animatable 3D human avatars from monocular video is a fundamental problem in computer vision with broad applications in AR/VR and digital content creation. Existing approaches typically couple parametric body models with neural rendering or 3D Gaussian splatting and optimize all body regions jointly from short videos, which often degrades fidelity in the visible areas. To overcome this limitation, we introduce FlexiAvatar, a unified framework that explicitly optimizes only the visible body regions, effectively eliminating artifacts arising from unobserved limbs. Our method integrates occlusion-robust SMPL-X tracking with part-specific residual refinement to capture high-frequency geometric and appearance details. To complete entirely unseen regions (e.g., back views), we leverage a diffusion-based approach to generate texture consistent with the observed appearance. Experiments on full-body (NeuMan, ZJU-MoCap, WildAvatar), upper/half-body (talk-show clips), and head-only (INSTA) inputs show that FlexiAvatar delivers consistently higher reconstruction quality, outperforming state-of-the-art methods by an average PSNR improvement of approximately 3% across datasets. Finally, by restricting optimization to observed regions, our method reduces the effective number of Gaussians that must be optimized and rendered, leading to reduced runtime and memory overhead in partial-visibility scenarios.

Yihalem Yimolal Tiruneh, Muhammad Salman Ali, Uyoung Jeong +5

UNIST, South Korea · University of Aberdeen, UK · University College London, UK +1

3D Gaussian Splatting 3D Reconstruction Monocular 3D Reconstruction 3D Avatar Generation 3D Human Reconstruction

Ask PDF

Abstract

3D Human Reconstruction

Explore similar work

Few TensoRF: Enhance the Few-shot on Tensorial Radiance Fields

GUSH3R: Everyone Everywhere All at Once as Gaussians

FlexiAvatar: Unified 3D Gaussian Human Avatars Under Arbitrary Body Visibility