Temporally consistent surface reconstruction of dynamic 3D objects from unstructured point cloud data remains challenging, especially for very long sequences. Existing methods either optimize deformations incrementally, risking drift and requiring long runtimes, or rely on complex learned models that demand category-specific training. We present Neu-PiG, a fast deformation optimization method based on a novel preconditioned latent-grid encoding that distributes spatial features parameterized on the position and normal direction of a keyframe surface. Our method encodes entire deformations across all time steps at various spatial scales into a multi-resolution latent grid, parameterized by the position and normal direction of a reference surface from a single keyframe. This latent representation is then augmented for time modulation and decoded into per-frame 6-DoF deformations via a lightweight multilayer perceptron (MLP). To achieve high-fidelity, drift-free surface reconstructions in seconds, we employ Sobolev preconditioning during gradient-based training of the latent space, completely avoiding the need for any explicit correspondences or further priors. Experiments across diverse human and animal datasets demonstrate that Neu-PiG outperforms state-the-art approaches, offering both superior accuracy and scalability to long sequences while running at least 60x faster than existing training-free methods and achieving inference speeds on the same order as heavy pretrained models.
Figures & tables
Figure 2: Overview of Neu-PiG. A reference surface Xtkey is first generated from the input point cloud at a keyframe Ptkey . Position and normal-direction-embedded latent features zp and zn are then sampled from multi-resolution preconditioned grids and combined with a time embedding γ(t) . A lightweight MLP ψ predicts per-frame 6-DoF deformations that are applied to Xtkey to reconstruct each frame X^t . Sobolev preconditioning in the latent space enforces spatial smoothness and temporal coherence across the sequence during optimization.
Figure 3: Qualitative reconstructions on DT4D. Neu-PiG preserves geometry and temporal consistency across complex animal motions.
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
Time ↓
DFAUST
CaDeX
0 3.68
0.941
0.730
0.013
0 3 s mi
DynoSurf
0 2.13
0.953
0.980
0.010
30 min
M2V
0 1.61
0.960
0.877
-
0 4 s
PDG
0 0.46
0.956
0.987
0.017
0 7 min
Ours †
0 0.40
0.967
0.989
0.008
0 8 s
Ours
0 0.39
0.968
0.989
0.009
32 s
Table 1: Quantitative comparison on DFAUST, AMA, and DT4D. Neu-PiG achieves best accuracy and temporal coherence while being 60 × faster than prior training-free methods.
Figure 4: Convergence of Chamfer Distance and Normal Consistency on DT4D, showing high-quality reconstructions in seconds.
Figure 5: Performance across sequence lengths on AMA. Neu-PiG maintains accuracy and stability as duration increases.
T
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
Time ↓
40
DynoSurf
0 4.64
0.857
0.686
0.096
0 35 min
PDG
0 0.66
0.923
0.971
0.042
0 28 min
Ours
0 0.53
0.947
0.981
0.019
0 47 s mi
60
DynoSurf
14.64
0.778
0.474
0.132
0 48 min
PDG
0 0.72
0.919
0.960
0.054
0 63 min
Ours
0 0.60
0.945
0.976
0.020
0 63 s mi
Table 2: Scalability on AMA sequences of increasing length. Neu-PiG remains stable and efficient, unlike PDG and DynoSurf.
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
Hash Encoding
1.23
0.903
0.947
0.045
w/o Preconditioning
0.98
0.955
0.958
0.036
w/o zn
0.91
0.968
0.965
0.036
w/o Liso
1.03
0.961
0.959
0.044
Single Level L=1
0.98
0.965
0.961
0.036
Ours
0.87
0.969
0.968
0.034
Table 3: Ablation on architectural components. Removing any module reduces accuracy, confirming their complementary roles.
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
Polynomial
0.46
0.951
0.987
0.019
Gaussian
0.77
0.945
0.979
0.046
Learned
0.50
0.950
0.983
0.022
Ours
0.44
0.951
0.989
0.017
Table 4: Comparison of time encoding strategies. Our Fourier encoding achieves the best accuracy and coherence.
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
δ
Constant
3.22
0.905
0.899
0.114
Linear
0.60
0.946
0.971
0.025
Exponential
0.53
0.946
0.982
0.030
Interpolated
0.64
0.945
0.972
0.046
Ours
0.53
0.947
0.981
0.025
ω(t)
Constant
3.27
0.905
0.898
0.114
Table 5: Ablation of stability terms δ and ω on AMA with T=40 . Our formulation yields best convergence and accuracy.
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
L=1
1.96
0.850
0.810
0.063
L=2
1.76
0.859
0.841
0.054
L=4
0.54
0.945
0.980
0.023
L=6
0.46
0.949
0.987
0.018
Ours ( L=8 )
0.44
0.951
0.989
0.017
L=10
0.45
0.951
0.989
0.021
Table 6: Ablation study on the number of grid levels in Gp , evaluated on the AMA dataset. Increasing the number of levels improves reconstruction quality up to L=8 , after which performance slightly degrades due to overfitting.
Figure 6: Qualitative analysis of the effect of the number of grid levels during optimization on reconstruction quality. Neu-PiG achieves accurate and temporally stable reconstructions even with few grid levels, while excessively increasing resolution leads to mild overfitting and loss of fine structural detail.
Figure 7: After optimization, we progressively restrict Neu-PiG to lmax grid levels and visualize the deformations, colored by a rigidity proxy. It shows that coarse levels capture smooth, global motion, whereas finer levels contribute localized, high-frequency detail, supporting the role of the multi-resolution latent grid.
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
λ1=0
0.58
0.928
0.977
0.022
λ1=0.08
0.45
0.950
0.988
0.020
Ours ( λ1=0.4 )
0.44
0.951
0.989
0.017
λ1=2
0.46
0.948
0.986
0.021
λ1=10
0.50
0.946
0.983
0.025
Table 7: Ablation study on the smoothness weight λ of the base grid level ( l=1 ), evaluated on the AMA dataset. Neu-PiG remains robust under moderate variations of λ , while extreme values reduce reconstruction quality.
Figure 8: Visualization of the learned latent space for different smoothness weights. We perform a PCA analysis on the latent vectors and display the first principal component as a spatial field.
Noise
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
0.25%
0.69
0.925
0.967
0.021
0.5%
1.34
0.866
0.853
0.026
1%
4.45
0.681
0.540
0.049
2%
21.71
0.517
0.247
0.086
Table 8: Ablation study on robustness to input noise with varying strengths based on the bounding box diagonal. Neu-PiG maintains stable reconstruction quality up to 0.5% noise, after which performance degrades as noise is captured in the surface geometry.
Table 9: Reconstruction performance at varying input point cloud resolutions. Neu-PiG consistently improves with higher input densities, closely matching the behavior of PDG while maintaining superior quality across all settings. In contrast, DynoSurf shows limited scalability and degrading performance at higher resolutions.
Figure 9: Extended qualitative results across diverse human and animal sequences. Neu-PiG consistently reconstructs temporally stable and detailed surfaces, even under large non-rigid deformations and challenging motion dynamics.
Figure 10: Extended visual results across diverse human and animal sequences. The surface normals of the deformed meshes are shown in comparison to ground truth surface normals.
Table 10: Ablation study on the temporal deformation network architecture, evaluated on the AMA dataset with 40 time steps per sequence. We vary latent dimensionality, number of time-encoding frequencies, and network capacity to assess their effect on reconstruction quality. Neu-PiG remains stable under most configurations, though extreme reductions in capacity or excessive latent dimensionality lead to degraded performance or overfitting. Our default configuration is shown in the last row.
Epochs
CD [×10−5]↓
NC ↑
F- 0.5%↑
Corr. ↓
75
Cayley
1.96
0.921
0.866
0.047
Exponential
1.62
0.924
0.886
0.043
Quaternions
1.59
0.923
0.884
0.044
125
Cayley
0.72
0.940
0.955
0.029
Exponential
0.68
0.941
0.961
0.027
Quaternions
0.65
0.942
0.965
0.028
Table 11: Comparison of different rotation parameterizations predicted by the deformation model. We evaluate quaternions, the Cayley transform, and the exponential map under identical optimization conditions. All representations achieve similar accuracy, while using quaternions leads to slightly faster convergence and improves performance with fewer epochs.
This paper addresses the problem of dynamic scene surface reconstruction using Gaussian Splatting (GS), aiming to recover temporally consistent geometry. While existing GS-based dynamic surface reconstruction methods can yield superior reconstruction, they are typically limited to either a single object or objects with only small deformations, struggling to maintain temporally consistent surface reconstruction of large deformations over time. We propose ``4DSurf'', a novel and unified framework for generic dynamic surface reconstruction that does not require specifying the number or types of objects in the scene, can handle large surface deformations and temporal inconsistency in reconstruction. The key innovation of our framework is the introduction of Gaussian deformations induced Signed Distance Function Flow Regularization that constrains the motion of Gaussians to align with the evolving surface. To handle large deformations, we introduce an Overlapping Segment Partitioning strategy that divides the sequence into overlapping segments with small deformations and incrementally passes geometric information across segments through the shared overlapping timestep. Experiments on two challenging dynamic scene datasets, Hi4D and CMU Panoptic, demonstrate that our method outperforms state-of-the-art surface reconstruction methods by 49% and 19% in Chamfer distance, respectively, and achieves superior temporal consistency under sparse-view settings.
This study addresses the partial-to-complete geometry reconstruction of deformable objects (DOs) from point-cloud observations toward precise DO manipulation. Recent DO reconstruction approaches often adopt implicit neural representations (INRs) to model continuous surfaces as well as capture structural variability. However, these methods typically rely on object-specific shape priors that improve training stability and limit generalization. To figure it out, we introduce ParCo-SDF, a two-stage partial-to-complete signed distance field (SDF) reconstruction framework consisting of temporal geometry encoding followed by FiLM-conditioned SDF prediction. The temporal encoder captures structural similarity across DO sequence, enabling prior-free stable training. FiLM-based conditioning preserves reconstruction expressivity while reducing network complexity. We evaluate the proposed method against a state-of-the-art DO surface reconstruction baseline on a rubber band manipulation dataset, demonstrating robust and high-fidelity reconstruction under severe occlusions.
Deokmin Hwang, Minseok Song, Daehyung Park
School of Computing, Korea Advanced Institute of Science and Technology, Korea
Dynamic Gaussian reconstruction requires compact scene features to support coordinated changes in geometry and appearance. A factorized grid provides efficient local queries, but accurate interpolation alone does not ensure that the resulting dynamic attributes remain compatible across space and time. We introduce DimFlow, a framework that learns these features through bidirectional interaction between a multi-resolution grid and a coordinate-based expert branch. Feature-level adversarial alignment connects their distributions, adaptive fusion transfers complementary information, and activation-masked residual feedback refines the grid during training. Four attribute experts organize deformation, rotation, opacity, and temporal information before Gaussian rendering. In the aggregate comparison across D-NeRF, HyperNeRF, and DAVIS, DimFlow achieves 39.73 dB PSNR, improving on 4D-GS by 1.92 dB while matching its reported 85 FPS rendering speed. The model occupies 18.9 MB, compared with 18.3 MB for 4D-GS. Component and attribute-expert ablations reveal complementary contributions from feature alignment, adaptive exchange, and temporal processing.
Gabriel Adineera, Jinoh Kim
Gabriel Adineera and Jinoh Kim are with Texas A&M University-Commerce, USA.