cs.CVNov 25, 2025

Material-informed Gaussian Splatting for 3D World Reconstruction in a Digital Twin

Authors: Andy Huynh, João Malheiro Silva, Holger Caesar, Tong Duy Son

Organizations: Siemens Digital Industries Software, 3001 Leuven, Belgium. · Dept. of Cognitive Robotics, Delft University of Technology, 2628 CD Delft, The Netherlands.

Abstract

3D reconstruction for Digital Twins often relies on LiDAR-based methods, which provide accurate geometry but lack the semantics and textures naturally captured by cameras. Traditional LiDAR-camera fusion approaches require complex calibration and still struggle with certain materials like glass, which are visible in images but poorly represented in point clouds. We propose a camera-only pipeline that reconstructs scenes using 3D Gaussian Splatting from multi-view images, extracts semantic material masks via vision models, converts Gaussian representations to mesh surfaces with projected material labels, and assigns physics-based material properties for accurate sensor simulation in modern graphics engines and simulators. This approach combines photorealistic reconstruction with physics-based material assignment, providing sensor simulation fidelity comparable to LiDAR-camera fusion while eliminating hardware complexity and calibration requirements. We validate our camera-only method using an internal dataset from an instrumented test vehicle, leveraging LiDAR as ground truth for reflectivity validation alongside image similarity metrics.

Figures & tables

Explore similar work

CardsList
  1. Transforming Omnidirectional RGB-LiDAR data into 3D Gaussian Splatting

    Mar 6, 2026Semin Bae, Hansol Lim, Jongseong Brad Choi3D Gaussian SplattingPanoramic 3D Reconstruction

  2. CADSplat: Sparse-View 3D Gaussian Splatting Aided by CAD Models for Robust, Photorealistic Digital-Twin Reconstruction

    Sep 16, 2026Kristof Overdulve, Lode Jorissen, Nick Michiels3D ReconstructionSparse-View 3D Reconstruction

  3. Structured-Li-GS: Structured 3D Gaussians Splatting with LiDAR Incorporation and Spatial Constraints

    Jun 25, 2026Huaiyuan Weng, Huibin Li, Chul Min Yeum3D Gaussian Splatting3D Reconstruction