Streaming 3D Reconstruction

Ask about this topic

Overview

Streaming 3D reconstruction estimates camera motion and scene geometry as frames arrive, producing point maps, meshes, or renderable Gaussian-splat scenes for robotics, mapping, and interactive viewing. Methods range from causal vision and geometry transformers and recurrent state models to SLAM pipelines with bundle adjustment, pose graphs, and incremental Gaussian or TSDF mapping; test-time training and external memories carry context across long sequences. To bound cost, systems compress or retrieve tokens, retain spatial anchors, and gate updates using novelty, confidence, or motion, while dynamic-scene methods add temporal deformation and semantic or object tracking. Evaluation on indoor RGB-D datasets such as 7-Scenes, ScanNet, and TUM, and outdoor driving sequences such as KITTI, measures pose drift, geometric accuracy, rendering quality, speed, and memory; the central challenge is preventing drift and forgetting without making updates too slow or unresponsive to scene changes.

Latest papers

CardsList

99 papers12 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

  1. StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression

    Apr 16, 2026Xuanyi Liu, Chunan Yu, Deyi Ji +63D ViTsStreaming 3D Reconstruction

    KOKONI 3D, Moxin Technology · Nanjing University of Science and Technology · University of Science and Technology of China +4

  2. LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstruction

    Apr 15, 2026Lin-Zhuo Chen, Jian Gao, Shangzhan Zhang +93D ViTsStreaming 3D Reconstruction

  3. ClipGStream: Clip-Stream Gaussian Splatting for Any Length and Any Motion Multi-View Dynamic Scene Reconstruction

    Apr 15, 2026Jie Liang, Jiahao Wu, Chao Wang +7Dynamic Gaussian Splatting3D Reconstruction

    Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University · Pengcheng Laboratory · Peking University +1

  4. Online3R: Online Learning for Consistent Sequential Reconstruction Based on Geometry Foundation Model

    Apr 10, 2026Shunkai Zhou, Zike Yan, Fei Xue +3Streaming 3D ReconstructionTest-Time Adaptation

    School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence · T-Stone Robotics Institute, The Chinese University of Hong Kong +3

  5. Incremental Semantics-Aided Meshing from LiDAR-Inertial Odometry and RGB Direct Label Transfer

    Apr 10, 2026Muhammad Affan, Ville Lehtola, George Vosselman3D Reconstruction3D Mesh Reconstruction

    Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, 7522 NB Enschede, Netherlands

  6. Fast Spatial Memory with Elastic Test-Time Training

    Apr 8, 2026Ziqiao Ma, Xueyang Yu, Haoyu Zhen +3Test-Time Training4D Reconstruction

    MIT-IBM Watson AI Lab · University of Michigan · University of Massachusetts Amherst

  7. Mem3R: Streaming 3D Reconstruction with Hybrid Memory

    Apr 8, 2026Changkun Liu, Jiezhi Yang, Zeman Li +4Streaming 3D ReconstructionMemory-Augmented Neural Networks

    Google · The Hong Kong University of Science and Technology · University of Southern California

  8. LiveStre4m: Feed-Forward Live Streaming of Novel Views from Unposed Multi-View Video

    Apr 8, 2026Pedro Quesado, Erkut Akdag, Yasaman Kashefbahrami +2Novel View SynthesisDynamic Novel View Synthesis

    AIMSGroup, Department of Electrical Engineering, Eindhoven University of Technology

  9. ZeD-MAP: Bundle Adjustment Guided Zero-Shot Depth Maps for Real-Time Aerial Imaging

    Apr 6, 2026Selim Ahmet Iz, Francesco Nex, Norman Kerle +2Aerial 3D Reconstruction3D Reconstruction

    German Aerospace Center (DLR), Institute of Space Research, Berlin, Germany · Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente, Enschede, The Netherlands

  10. DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale

    Apr 1, 2026Sicheng Zuo, Zixun Xie, Wenzhao Zheng +6End-to-End Autonomous DrivingAutonomous Driving Planning

    Tsinghua University · Xiaomi EV · Peking University +1

  11. GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates

    Mar 30, 2026Youngjoong Kwon, Yao He, Heejung Choi +43D Human ReconstructionNovel View Synthesis

    Stanford University

  12. MoRGS: Efficient Per-Gaussian Motion Reasoning for Streamable Dynamic 3D Scenes

    Mar 26, 2026Wonjoon Lee, Sungmin Woo, Donghyeong Kim +3Dynamic Scene Reconstruction4D Reconstruction

    Yonsei University · Electronics and Telecommunications Research Institute (ETRI)

  13. SLARM: Streaming and Language-Aligned Reconstruction Model for Dynamic Scenes

    Mar 24, 2026Zhicheng Qiu, Jiarui Meng, Tong-an Luo +4Dynamic Scene ReconstructionStreaming 3D Reconstruction

    Huawei Technologies Ltd.

  14. PAS3R: Pose-Adaptive Streaming 3D Reconstruction for Long Video Sequences

    Mar 22, 2026Lanbo Xu, Liang Guo, Caigui Jiang +1Streaming 3D ReconstructionMonocular 3D Reconstruction

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · University of East Anglia, Norwich, NR47TJ, United Kingdom

  15. RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction

    Mar 21, 2026Feiran Wang, Zezhou Shang, Gaowen Liu +1Motion SegmentationDynamic Scene Reconstruction

    University of Illinois Chicago · Cisco Research

  16. FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction

    Mar 19, 2026Seonghyun Jin, Jong Chul YeStreaming 3D ReconstructionKalman Filtering

    Kim Jaechul Graduate School of AI, KAIST, Seoul, South Korea

  17. STAC: Plug-and-Play Spatio-Temporal Aware Cache Compression for Streaming 3D Reconstruction

    Mar 18, 2026Runze Wang, Yuxuan Song, Youcheng Cai +1Streaming 3D ReconstructionKV-Cache Compression

    University of Science and Technology of China

  18. Does it matter which Gaussians you pick in 4D Gaussian streaming?

    Mar 18, 2026Ashim Dahal, Rabab Abdelfattah, Nick Rahimi3D Gaussian Splatting4D Reconstruction

    School of Computing Sciences and Computer Engineering University of Southern Mississippi, Hattiesburg, MS

  19. M^3: Dense Matching Meets Multi-View Foundation Models for Monocular Gaussian Splatting SLAM

    Mar 17, 2026Kerui Ren, Guanghao Li, Changjian Jiang +7Visual SLAMStreaming 3D Reconstruction

    Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Fudan University +5

  20. MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction

    Mar 16, 2026Jiacheng Dong, Huan Li, Sicheng Zhou +3Streaming 3D Reconstruction3D Reconstruction

    Zhejiang University · Institute for AI Industry Research, Tsinghua University

  21. SSR: A Training-Free Approach for Streaming 3D Reconstruction

    Mar 16, 2026Hui Deng, Yuxin Mao, Yuxin He +1Streaming 3D Reconstruction3D Reconstruction

    Shool of Electronics and Information, Northwestern Polytechnical University and Shaanxi Key Laboratory of Information Acquisition and Processing, Xi’an, China.

  22. S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction

    Mar 15, 2026Renhe Zhang, Yuyang Tan, Jingyu Gong +4Streaming 3D Reconstruction3D Scene Understanding

    East China Normal University · Shanghai Artificial Intelligence Laboratory

  23. GelSphere: An Omnidirectional Rolling Vision-Based Tactile Sensor for Online 3D Reconstruction and Normal Force Estimation

    Mar 14, 2026Seoyeon Lee, Mohammad Amin Mirzaee, Wenzhen YuanVision-Based Tactile SensingTactile Sensing

    University of Illinois at Urbana-Champaign, Champaign, IL, USA

  24. OmniStream: Mastering Perception, Reconstruction and Action in Continuous Streams

    Mar 12, 2026Yibin Yan, Jilan Xu, Shangzhe Di +2Streaming Video UnderstandingVision Foundation Models

    School of Artificial Intelligence, SJTU · Shanghai Innovation Institute · VGG, Oxford

  25. ReCoSplat: Online Feed-Forward Gaussian Splatting via Render-and-Compare

    Mar 10, 2026Freeman Cheng, Botao Ye, Xueting Li +33D Gaussian SplattingStreaming 3D Reconstruction

    University of California, Merced · ETH Zürich · NVIDIA +2

  26. GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System

    Mar 10, 2026Zhiye Tang, Qiudan Zhang, Lei Zhang +3Streaming 3D Reconstruction3D Gaussian Splatting

    College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, China. · Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong SAR, China. · School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China.

  27. FrameVGGT: Coherence-Preserving Memory for Bounded Streaming Geometry

    Mar 8, 2026Zhisong Xu, Takeshi OishiVideo Depth Estimation3D ViTs

    Institute of Industrial Science, The University of Tokyo, Tokyo, Japan

  28. OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer

    Mar 6, 2026Si-Yu Lu, Po-Ting Chen, Hui-Che Hsu +3Streaming 3D Reconstruction3D ViTs

    National Taiwan University · National Taiwan University of Science and Technology

  29. Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setups

    Mar 5, 2026Leif Van Holland, Domenic Zingsheim, Mana Takhsha +4Image InpaintingStreaming 3D Reconstruction

    University of Bonn, Germany

  30. ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training

    Mar 4, 2026Haian Jin, Rundi Wu, Tianyuan Zhang +43D ReconstructionMulti-View 3D Reconstruction

    Google DeepMind · Cornell University · Massachusetts Institute of Technology