Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences

Julian Kaltheuner; Hannah Dröge; Markus Plack; Patrick Stotko; Reinhard Klein

cs.CVFeb 25, 2026

Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences

Authors: Julian Kaltheuner, Hannah Dröge, Markus Plack, Patrick Stotko, Reinhard Klein

Organizations: University of Bonn

Abstract

Temporally consistent surface reconstruction of dynamic 3D objects from unstructured point cloud data remains challenging, especially for very long sequences. Existing methods either optimize deformations incrementally, risking drift and requiring long runtimes, or rely on complex learned models that demand category-specific training. We present Neu-PiG, a fast deformation optimization method based on a novel preconditioned latent-grid encoding that distributes spatial features parameterized on the position and normal direction of a keyframe surface. Our method encodes entire deformations across all time steps at various spatial scales into a multi-resolution latent grid, parameterized by the position and normal direction of a reference surface from a single keyframe. This latent representation is then augmented for time modulation and decoded into per-frame 6-DoF deformations via a lightweight multilayer perceptron (MLP). To achieve high-fidelity, drift-free surface reconstructions in seconds, we employ Sobolev preconditioning during gradient-based training of the latent space, completely avoiding the need for any explicit correspondences or further priors. Experiments across diverse human and animal datasets demonstrate that Neu-PiG outperforms state-the-art approaches, offering both superior accuracy and scalability to long sequences while running at least 60x faster than existing training-free methods and achieving inference speeds on the same order as heavy pretrained models.

Figures & tables

Figure 2: Overview of Neu-PiG. A reference surface $\mathcal{X}_{t_{\mathrm{key}}}$ is first generated from the input point cloud at a keyframe $\mathcal{P}_{t_{\mathrm{key}}}$ . Position and normal-direction-embedded latent features $\bm{z}_{\mathrm{p}}$ and $\bm{z}_{\mathrm{n}}$ are then sampled from multi-resolution preconditioned grids and combined with a time embedding $\bm{\gamma}(t)$ . A lightweight MLP $\psi$ predicts per-frame 6-DoF deformations that are applied to $\mathcal{X}_{t_{\mathrm{key}}}$ to reconstruct each frame $\hat{\mathcal{X}}_{t}$ . Sobolev preconditioning in the latent space enforces spatial smoothness and temporal coherence across the sequence during optimization.

Figure 3: Qualitative reconstructions on DT4D. Neu-PiG preserves geometry and temporal consistency across complex animal motions.

		CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$	Time $\downarrow$
DFAUST	CaDeX	0 3.68	0.941	0.730	0.013	0 3 s mi
	DynoSurf	0 2.13	0.953	0.980	0.010	30 min
	M2V	0 1.61	0.960	0.877	-	0 4 s
	PDG	0 0.46	0.956	0.987	0.017	0 7 min
	Ours $\dagger$	0 0.40	0.967	0.989	0.008	0 8 s
	Ours	0 0.39	0.968	0.989	0.009	32 s

Table 1: Quantitative comparison on DFAUST, AMA, and DT4D. Neu-PiG achieves best accuracy and temporal coherence while being 60

\times

faster than prior training-free methods.

Figure 4: Convergence of Chamfer Distance and Normal Consistency on DT4D, showing high-quality reconstructions in seconds.

Figure 5: Performance across sequence lengths on AMA. Neu-PiG maintains accuracy and stability as duration increases.

$T$		CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$	Time $\downarrow$
40	DynoSurf	0 4.64	0.857	0.686	0.096	0 35 min
	PDG	0 0.66	0.923	0.971	0.042	0 28 min
	Ours	0 0.53	0.947	0.981	0.019	0 47 s mi
60	DynoSurf	14.64	0.778	0.474	0.132	0 48 min
	PDG	0 0.72	0.919	0.960	0.054	0 63 min
	Ours	0 0.60	0.945	0.976	0.020	0 63 s mi

Table 2: Scalability on AMA sequences of increasing length. Neu-PiG remains stable and efficient, unlike PDG and DynoSurf.

	CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
Hash Encoding	1.23	0.903	0.947	0.045
w/o Preconditioning	0.98	0.955	0.958	0.036
w/o $\bm{z}_{\mathrm{n}}$	0.91	0.968	0.965	0.036
w/o $\mathcal{L}_{\mathrm{iso}}$	1.03	0.961	0.959	0.044
Single Level $L=1$	0.98	0.965	0.961	0.036
Ours	0.87	0.969	0.968	0.034

Table 3: Ablation on architectural components. Removing any module reduces accuracy, confirming their complementary roles.

	CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
Polynomial	0.46	0.951	0.987	0.019
Gaussian	0.77	0.945	0.979	0.046
Learned	0.50	0.950	0.983	0.022
Ours	0.44	0.951	0.989	0.017

Table 4: Comparison of time encoding strategies. Our Fourier encoding achieves the best accuracy and coherence.

		CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
$\delta$	Constant	3.22	0.905	0.899	0.114
	Linear	0.60	0.946	0.971	0.025
	Exponential	0.53	0.946	0.982	0.030
	Interpolated	0.64	0.945	0.972	0.046
	Ours	0.53	0.947	0.981	0.025
$\omega(t)$	Constant	3.27	0.905	0.898	0.114

Table 5: Ablation of stability terms

\delta

and

\omega

on AMA with

{T=40}

. Our formulation yields best convergence and accuracy.

	CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
$L=1$	1.96	0.850	0.810	0.063
$L=2$	1.76	0.859	0.841	0.054
$L=4$	0.54	0.945	0.980	0.023
$L=6$	0.46	0.949	0.987	0.018
Ours ( $L=8$ )	0.44	0.951	0.989	0.017
$L=10$	0.45	0.951	0.989	0.021

Table 6: Ablation study on the number of grid levels in

\mathcal{G}_{\mathrm{p}}

, evaluated on the AMA dataset. Increasing the number of levels improves reconstruction quality up to

{L=8}

, after which performance slightly degrades due to overfitting.

Figure 6: Qualitative analysis of the effect of the number of grid levels during optimization on reconstruction quality. Neu-PiG achieves accurate and temporally stable reconstructions even with few grid levels, while excessively increasing resolution leads to mild overfitting and loss of fine structural detail.

Figure 7: After optimization, we progressively restrict Neu-PiG to $l_{\text{max}}$ grid levels and visualize the deformations, colored by a rigidity proxy. It shows that coarse levels capture smooth, global motion, whereas finer levels contribute localized, high-frequency detail, supporting the role of the multi-resolution latent grid.

	CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
$\lambda^{1}=0$	0.58	0.928	0.977	0.022
$\lambda^{1}=0.08$	0.45	0.950	0.988	0.020
Ours ( $\lambda^{1}=0.4$ )	0.44	0.951	0.989	0.017
$\lambda^{1}=2$	0.46	0.948	0.986	0.021
$\lambda^{1}=10$	0.50	0.946	0.983	0.025

Table 7: Ablation study on the smoothness weight

\lambda

of the base grid level (

{l=1}

), evaluated on the AMA dataset. Neu-PiG remains robust under moderate variations of

\lambda

, while extreme values reduce reconstruction quality.

Figure 8: Visualization of the learned latent space for different smoothness weights. We perform a PCA analysis on the latent vectors and display the first principal component as a spatial field.

Noise	CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
$0.25\%$	0.69	0.925	0.967	0.021
$0.5\%$	1.34	0.866	0.853	0.026
$1\%$	4.45	0.681	0.540	0.049
$2\%$	21.71	0.517	0.247	0.086

Table 8: Ablation study on robustness to input noise with varying strengths based on the bounding box diagonal. Neu-PiG maintains stable reconstruction quality up to

0.5\%

noise, after which performance degrades as noise is captured in the surface geometry.

$\mathopen{}\mathclose{{\left\|\mathcal{P}_{t}}}\right\|$		CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
0 2500	DynoSurf	1.56	0.896	0.862	0.041
	PDG	0.79	0.922	0.948	0.037
	Ours	0.75	0.930	0.953	0.022
0 5000	DynoSurf	1.01	0.918	0.921	0.044
	PDG	0.47	0.939	0.985	0.030
	Ours	0.44	0.950	0.988	0.018

Table 9: Reconstruction performance at varying input point cloud resolutions. Neu-PiG consistently improves with higher input densities, closely matching the behavior of PDG while maintaining superior quality across all settings. In contrast, DynoSurf shows limited scalability and degrading performance at higher resolutions.

Figure 9: Extended qualitative results across diverse human and animal sequences. Neu-PiG consistently reconstructs temporally stable and detailed surfaces, even under large non-rigid deformations and challenging motion dynamics.

Figure 10: Extended visual results across diverse human and animal sequences. The surface normals of the deformed meshes are shown in comparison to ground truth surface normals.

$\mathopen{}\mathclose{{\left\|\psi}}\right\|$	$\mathopen{}\mathclose{{\left\|\bm{z}_{\mathrm{p}}}}\right\|$	$\mathopen{}\mathclose{{\left\|\bm{z}_{\mathrm{n}}}}\right\|$	$\mathopen{}\mathclose{{\left\|\bm{\gamma}}}\right\|$	CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
512	30	2	2	0.63	0.945	0.970	0.027
512	30	2	32	0.64	0.943	0.973	0.040
512	30	1	8	0.55	0.946	0.979	0.026
512	30	8	8	0.57	0.947	0.978	0.026
512	8	2	8	0.61	0.945	0.975	0.023
512	120	2	8	0.59	0.947	0.978	0.032

Table 10: Ablation study on the temporal deformation network architecture, evaluated on the AMA dataset with 40 time steps per sequence. We vary latent dimensionality, number of time-encoding frequencies, and network capacity to assess their effect on reconstruction quality. Neu-PiG remains stable under most configurations, though extreme reductions in capacity or excessive latent dimensionality lead to degraded performance or overfitting. Our default configuration is shown in the last row.

Epochs		CD $[\times 10^{-5}]$ $\downarrow$	NC $\uparrow$	F- $0.5\%$ $\uparrow$	Corr. $\downarrow$
75	Cayley	1.96	0.921	0.866	0.047
	Exponential	1.62	0.924	0.886	0.043
	Quaternions	1.59	0.923	0.884	0.044
125	Cayley	0.72	0.940	0.955	0.029
	Exponential	0.68	0.941	0.961	0.027
	Quaternions	0.65	0.942	0.965	0.028

Table 11: Comparison of different rotation parameterizations predicted by the deformation model. We evaluate quaternions, the Cayley transform, and the exponential map under identical optimization conditions. All representations achieve similar accuracy, while using quaternions leads to slightly faster convergence and improves performance with fewer epochs.

Dynamic Scene Reconstruction

Dynamic scene reconstruction builds time-varying 3D models of moving objects and environments from video or multi-view images.

See all 288 papers

4DSurf: High-Fidelity Dynamic Scene Surface Reconstruction
Mar 30, 2026Renjie Wu, Hongdong Li, Jose M. Alvarez +1Dynamic Scene Reconstruction 4D Reconstruction
Australian National University · Amazon · NVIDIA
ParCo-SDF: Learning Prior-Free Partial-to-Complete Signed Distance Fields of Deformable Objects
May 28, 2026Deokmin Hwang, Minseok Song, Daehyung ParkImplicit Neural Representations 3D Reconstruction
School of Computing, Korea Advanced Institute of Science and Technology, Korea
DimFlow: Bidirectional Grid-Expert Learning for Dynamic Gaussian Reconstruction
Oct 26, 2026Gabriel Adineera, Jinoh KimDynamic Scene Reconstruction 4D Reconstruction
Gabriel Adineera and Jinoh Kim are with Texas A&M University-Commerce, USA.

Mar 30, 2026J/K move · Enter open · S save

4DSurf: High-Fidelity Dynamic Scene Surface Reconstruction

Renjie Wu, Hongdong Li, Jose M. Alvarez, Miaomiao Liu

Australian National University · Amazon · NVIDIA

Dynamic Scene Reconstruction 4D Reconstruction Deformable Shape Reconstruction 3D Gaussian Splatting

Open paper AskPDF arXiv

This paper addresses the problem of dynamic scene surface reconstruction using Gaussian Splatting (GS), aiming to recover temporally consistent geometry. While existing GS-based dynamic surface reconstruction methods can yield superior reconstruction, they are typically limited to either a single object or objects with only small deformations, struggling to maintain temporally consistent surface reconstruction of large deformations over time. We propose ``4DSurf'', a novel and unified framework for generic dynamic surface reconstruction that does not require specifying the number or types of objects in the scene, can handle large surface deformations and temporal inconsistency in reconstruction. The key innovation of our framework is the introduction of Gaussian deformations induced Signed Distance Function Flow Regularization that constrains the motion of Gaussians to align with the evolving surface. To handle large deformations, we introduce an Overlapping Segment Partitioning strategy that divides the sequence into overlapping segments with small deformations and incrementally passes geometric information across segments through the shared overlapping timestep. Experiments on two challenging dynamic scene datasets, Hi4D and CMU Panoptic, demonstrate that our method outperforms state-of-the-art surface reconstruction methods by 49% and 19% in Chamfer distance, respectively, and achieves superior temporal consistency under sparse-view settings.

Abstract

Dynamic Scene Reconstruction

Explore similar work

4DSurf: High-Fidelity Dynamic Scene Surface Reconstruction

ParCo-SDF: Learning Prior-Free Partial-to-Complete Signed Distance Fields of Deformable Objects

DimFlow: Bidirectional Grid-Expert Learning for Dynamic Gaussian Reconstruction