FluidGaussian: Propagating Simulation-Based Uncertainty Toward Functionally-Intelligent 3D Reconstruction
Authors: Yuqiu Liu, Jialin Song, Marissa Ramirez de Chanlatte, Rochishnu Chowdhury, Rushil Paresh Desai, Wuyang Chen, Daniel Martin, Michael W. Mahoney
Organizations: Simon Fraser University · Lawrence Berkeley National Lab · University of California, Berkeley · International Computer Science Institute
Real objects that inhabit the physical world follow physical laws and thus behave plausibly during interaction with other physical objects. However, current methods that perform 3D reconstructions of real-world scenes from multi-view 2D images optimize primarily for visual fidelity, i.e., they train with photometric losses and reason about uncertainty in the image or representation space. This appearance-centric view overlooks body contacts and couplings, conflates function-critical regions (e.g., aerodynamic or hydrodynamic surfaces) with ornamentation, and reconstructs structures suboptimally, even when physical regularizers are added. All these can lead to unphysical and implausible interactions. To address this, we consider the question: How can 3D reconstruction become aware of real-world interactions and underlying object functionality, beyond visual cues? To answer this question, we propose FluidGaussian, a plug-and-play method that tightly couples geometry reconstruction with ubiquitous fluid-structure interactions to assess surface quality at high granularity. We define a simulation-based uncertainty metric induced by fluid simulations and integrate it with active learning to prioritize views that improve both visual and physical fidelity. In an empirical evaluation on NeRF Synthetic (Blender), Mip-NeRF 360, and DrivAerNet++, our FluidGaussian method yields up to +8.6% visual PSNR (Peak Signal-to-Noise Ratio) and -62.3% velocity divergence during fluid simulations. Our code is available at https://github.com/delta-lab-ai/FluidGaussian.
Figures & tables
Figure 1: When only visual fidelity is optimized and physical awareness is excluded [ 46 , 28 ] (left column), reconstructions may look correct visually (top), and yet can fail functionally : fluid simulation exhibits excessive velocity-field divergence (defined in Section 3.2 ), a deficiency that worsens from low to high turbulence (from middle to bottom). With FluidGaussian (right column), both visual fidelity and simulation fidelity are better preserved.
Figure 2: Overview of FluidGaussian. To introduce physics-awareness into 3D reconstruction: (1) start from K candidate camera poses proposed by vision-based NBV methods [ 46 , 28 ] ; and (2) evaluate a physics-informed uncertainty induced from fluid-structure simulations (Section 3.2 ) and select the best next camera pose (Section 3.3 ). The blue block denotes the fluid.
Figure 3: To make our physical uncertainty statistically meaningful, FluidGaussian measures over individual fluid simulations from five initial conditions (i.e., five individual simulations). Blue blocks denote fluids. We use a Y-up coordinate system.
Figure 4: 3D Gaussian accumulates divergence introduced by fluid particles in its neighborhood. Voxelized rigid particles (from 3D Gaussians) are omitted.
Figure 5: Quantification of function-critical region of DrivAerNet++ [ 15 ] : a) Simulation on the −ex direction to quantify functional-critical regions; b) Examples on function-critical regions in test set. Red arrow shows the direction of flow in the simulation. The rightmost figure indicates the rear of the vehicle.
Figure 6: Visualization of the object from Blender [ 40 ] , MipNeRF360 [ 3 ] , and DrivAerNet++ [ 15 ] . Each column represents different methods, from left to right: ActiveNeRF, ActiveNeRF + FluidGaussian, FisherRF, FisherRF + FluidGaussian, ground truth (GT).
Method
(a) Blender
(b) MipNeRF360
(c) DrivAerNet++
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
ActiveNeRF
21.1525
0.8527
0.1397
12.9245
0.4331
0.5036
17.3840
0.9079
0.1595
+FluidGaussian
21.7642
0.8613
0.1230
12.4790
0.4248
0.5067
18.8727
0.9297
0.1269
FisherRF
23.4230
0.8761
0.1070
15.3237
0.4704
0.4561
20.1127
0.9339
0.1250
+FluidGaussian
24.7433
0.8907
0.0923
15.5502
0.4717
0.4522
21.0733
0.9462
0.1047
Table 1: Visual reconstruction results on Blender [ 40 ] , MipNeRF360 [ 3 ] , and DrivAerNet++ [ 15 ] datasets. Adding FluidGaussian improves visual quality in most cases, with the largest gains on DrivAerNet++ [ 15 ] when using ActiveNeRF [ 46 ] as the baseline.
Dataset
Blender
MipNeRF360
DrivAerNet++
Training
Resolution
800×800
1600×1040
1600×900
Test views
200
87
100
Opacity ( α ) threshold
0.3
0.4
0.25
Iterations
30000
Initial views
2
Table 2: Training and simulation settings for all datasets.
Datasets
Blender
MipNeRF360
DrivAerNet++
Empty Scene
5.67×10−5
0.0010
4.34×10−4
Perfect Geometry
0.0058
0.0014
0.0371
ActiveNeRF
0.0262
0.0130
0.0894
+FluidGaussian
0.0145
0.0049
0.0801
FisherRF
0.0144
0.0025
0.0609
+FluidGaussian
0.0124
0.0022
0.0608
Table 3: Comparison of divergence ( Dˉ↓ ) after optimization. “Empty Scene”: simulation without a rigid body. “Perfect Geometry”: geometry trained with at least 20 views. Value differences in empty-scene divergence are caused by different spatial domains.
Figure 7: PSNR and divergence results on Blender [ 40 ] . FluidGaussian mitigates physical defects increasingly as the fluid transitions from laminar to turbulent. As the fluid velocity rises and the Reynolds number Re increases, appearance-only reconstructions exhibit markedly amplified physical errors, whereas FluidGaussian largely preserves physical fidelity and interaction plausibility.
DrivAerNet++
PSNR ↑
SSIM ↑
LPIPS ↓
Divergence ↓
ActiveNeRF
15.6770
0.8843
0.1896
0.0316
+FluidGaussian
16.8897
0.9086
0.1571
0.0231
FisherRF
18.2988
0.9206
0.1472
0.0213
+FluidGaussian
18.5436
0.9297
0.1312
0.0182
Table 4: Visual and physics-based measures on function-critical regions of the DrivAerNet++ dataset. Adding FluidGaussian gives better reconstruction quality and lower divergence.
Method (fluid direction)
PSNR ↑
SSIM ↑
LPIPS ↓
ex
24.3862
0.8857
0.0973
−ex
24.5565
0.8875
0.0949
ez
24.0015
0.8842
0.0985
−ez
24.1853
0.8836
0.0986
ey
24.2303
0.8851
0.0985
FisherRF [ 28 ]
23.4230
0.8761
0.1070
Table 5: Results on Blender [ 40 ] under different initial conditions.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Blender
Mip
DrivAer
ActiveNeRF + Random
21.547
12.415
18.770
ActiveNeRF + FluidGaussian
21.764
12.479
18.873
FisherRF + Random
23.631
15.326
20.022
FisherRF + FluidGaussian
24.743
15.550
21.073
Appendix
Table 6: Comparison between random view selection and FluidGaussian under the same reconstruction baseline. PSNR is reported.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
ActiveNeRF
24.828
0.961
0.064
ActiveNeRF + FluidGaussian
25.653
0.960
0.063
FisherRF
27.604
0.974
0.041
FisherRF + FluidGaussian
27.807
0.976
0.034
Appendix
Table 7: Comparison between the baseline and FluidGaussian on 7 ShapeNet objects.
Method
Chamfer Distance ↓
ActiveNeRF
0.0103
ActiveNeRF + FluidGaussian
0.0055
FisherRF
0.0165
FisherRF + FluidGaussian
0.0046
Appendix
Table 8: Chamfer Distance comparison on the Blender dataset. Lower is better.
Recent advances in Gaussian Splatting have enabled fast, high-fidelity 3D scene generation, yet these methods remain purely visual and lack an understanding of how shapes behave in the physical world. We introduce Physics-Guided 3D Gaussian Splatting (PG-3DGS), a framework that couples differentiable physics simulation with 3D Gaussian representations to generate 3D structures satisfying physics functionalities. By allowing physical objectives to guide the shape optimization process alongside visual losses, our approach produces geometries that are not only photometrically accurate but also physically functional. The model learns to adjust shapes so that the generated objects exhibit physically meaningful behaviors, for example, teapots that can pour and airplanes that can generate lift, without sacrificing visual quality. Experiments on pouring and aerodynamic lift tasks show that PG-3DGS improves physical functionality while preserving visual quality. In addition to simulation gains, bench-top physical lift tests with 3D-printed aircraft (Cessna, B-2 Spirit, and paper plane) under identical airflow conditions show higher scale-measured lift for PG-3DGS, generated structures than an appearance-matching baseline in all three cases. Our unified framework connects appearance-based reconstruction with physics-based reasoning, enabling end-to-end generation of 3D structures that both look realistic and function correctly.
Zachary Lee, Maxwell Jacobson, Yexiang Xue
Department of Computer Science, Purdue University, West Lafayette, 47907, IN, United States
We present GaussFusion, a novel approach for improving 3D Gaussian splatting (3DGS) reconstructions in the wild through geometry-informed video generation. GaussFusion mitigates common 3DGS artifacts, including floaters, flickering, and blur caused by camera pose errors, incomplete coverage, and noisy geometry initialization. Unlike prior RGB-based approaches limited to a single reconstruction pipeline, our method introduces a geometry-informed video-to-video generator that refines 3DGS renderings across both optimization-based and feed-forward methods. Given an existing reconstruction, we render a Gaussian primitive video buffer encoding depth, normals, opacity, and covariance, which the generator refines to produce temporally coherent, artifact-free frames. We further introduce an artifact synthesis pipeline that simulates diverse degradation patterns, ensuring robustness and generalization. GaussFusion achieves state-of-the-art performance on novel-view synthesis benchmarks, and an efficient variant runs in real time at 15 FPS while maintaining similar performance, enabling interactive 3D applications.
3D Gaussian Splatting (3DGS) has recently enabled real-time novel view synthesis with impressive quality. However, it struggles to recover accurate surfaces under limited viewpoints and due to the inherent irregularity of Gaussian primitives. The resulting geometric errors are notoriously difficult to correct manually. To address these issues, we propose Gaussian Sculpting, a fully differentiable end-to-end framework for high-quality surface reconstruction. Our key insight is to anchor Gaussians onto an evolving differentiable surface, allowing them to guide signed distance field (SDF) optimization instead of extracting the surface only during post-processing. To enable stable gradient isolation during joint optimization, we design a bi-level training strategy in which the outer loop optimizes the geometry represented by the SDF, while the inner loop updates the Gaussians with the geometry fixed. We further impose constraints on Gaussian parameters to ensure consistency with the underlying surface, thereby improving both geometric and appearance fidelity during optimization. In addition, we introduce a multi-resolution subdivision scheme based on octree-like partitioning to preserve fine details while reducing memory consumption. Experiments on object-level scenes demonstrate that our method effectively removes redundant surfaces, recovers missing structures caused by limited viewpoints, and achieves strong reconstruction quality even at relatively low resolutions.
Ke Jiaxin, Juncheng Liu, Yi Wang +4
Dalian University of Technology, China · Massey University, New Zealand · Peking University, China +1