3D Gaussian Splatting

Also known as 3DGS

3D Gaussian Splatting represents a scene with many small, positioned and oriented Gaussian primitives whose color and opacity are combined by a differentiable renderer. Methods either optimize these primitives from posed multi-view images or predict them with feed-forward networks, often using depth, stereo, or pretrained vision-model features to improve sparse-view reconstruction. Work also extends the representation to changing scenes, avatars, semantic mapping, and applications such as SLAM, while developing pruning, compression, and faster rasterization for limited hardware. A central challenge is balancing convincing novel views and real-time rendering against accurate, stable geometry, especially when views are sparse, cameras are uncertain, or objects move.

Latest papers

1,028 papers116 papers in the last four weeks, up 3% on the four weeks before. 0.7% of all new papers.

CATSplat: Context-Aware Transformer with Spatial Guidance for Generalizable 3D Gaussian Splatting from A Single-View Image
Dec 17, 2024Wonseok Roh, Hwanhee Jung, Jong Wook Kim +63D Gaussian Splatting 3D ViTs
Korea University · Google · Purdue University
SparseLGS: Sparse View Language Embedded Gaussian Splatting
Dec 3, 2024Jun Hu, Zhang Chen, Zhong Li +23D Gaussian Splatting Language-Embedded 3DGS
School of Mathematical Science, University of Science and Technology of China · Meta · Apple Inc. +1
GSurf: Learning Signed Distance Fields from Splatting Opaque Gaussians for High-quality 3D Reconstruction
Nov 24, 2024Baixin Xu, Jiangbei Hu, Jiaze Li +13D Gaussian Splatting Signed Distance Functions
College of Computing and Data Science, Nanyang Technological University · School of Software, Dalian University of Technology
MVGS: Multi-view Regulated Gaussian Splatting for Novel View Synthesis
Oct 2, 2024Xiaobiao Du, Yida Wang, Xin YuMulti-View Consistency 3D Gaussian Splatting
University of Technology Sydney · Li Auto Inc. · Australian Institute for Machine Learning, Adelaide University
MiraGe: Editable 2D Images using Gaussian Splatting
Oct 2, 2024Joanna Waczyńska, Tomasz Szczepanik, Piotr Borycki +33D Gaussian Splatting Image Editing
1Jagiellonian University · University of Cambridge
2DGH: 2D Gaussian-Hermite Splatting for High-quality Rendering and Better Geometry Features
Aug 30, 2024Ruihan Yu, Tianyu Huang, Jingwang Ling +13D Gaussian Splatting 3D Reconstruction
Department of Physics, Tsinghua University · School of software and BNRist, Tsinghua University
MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors
Jun 1, 2024Qingming Liu, Yuan Liu, Jiepeng Wang +43D Gaussian Splatting Dynamic Gaussian Splatting
City University of HongKong · CUHK(SZ) · HKUST +2
SparseGS: Sparse View Synthesis using 3D Gaussian Splatting
Nov 30, 2023Haolin Xiong, Sairisheek Muttukuru, Hanyuan Xiao +43D Gaussian Splatting Novel View Synthesis
University of California, Los Angeles · USC Institute for Creative Technologies · University of Southern California

Dec 17, 2024J/K move · Enter open · S save

CATSplat: Context-Aware Transformer with Spatial Guidance for Generalizable 3D Gaussian Splatting from A Single-View Image

Wonseok Roh, Hwanhee Jung, Jong Wook Kim, Seunggwan Lee+5

Korea University · Google · Purdue University

3D Gaussian Splatting 3D ViTs 3D Reconstruction Single-View 3D Reconstruction Feed-Forward 3D Reconstruction

Open paper AskPDF arXiv

Recently, generalizable feed-forward methods based on 3D Gaussian Splatting have gained significant attention for their potential to reconstruct 3D scenes using finite resources. These approaches create a 3D radiance field, parameterized by per-pixel 3D Gaussian primitives, from just a few images in a single forward pass. However, unlike multi-view methods that benefit from cross-view correspondences, 3D scene reconstruction with a single-view image remains an underexplored area. In this work, we introduce CATSplat, a novel generalizable transformer-based framework designed to break through the inherent constraints in monocular settings. First, we propose leveraging textual guidance from a visual-language model to complement insufficient information from a single image. By incorporating scene-specific contextual details from text embeddings through cross-attention, we pave the way for context-aware 3D scene reconstruction beyond relying solely on visual cues. Moreover, we advocate utilizing spatial guidance from 3D point features toward comprehensive geometric understanding under single-view settings. With 3D priors, image features can capture rich structural insights for predicting 3D Gaussians without multi-view techniques. Extensive experiments on large-scale datasets demonstrate the state-of-the-art performance of CATSplat in single-view 3D scene reconstruction with high-quality novel view synthesis.