AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environments

Xuzhi Wang; Xinran Wu; Song Wang; Lingdong Kong; Ziping Zhao

cs.CVMar 26, 2026

AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environments

Authors: Xuzhi Wang, Xinran Wu, Song Wang, Lingdong Kong, Ziping Zhao

Organizations: Tianjin Normal University · Zhejiang University · National University of Singapore

Abstract

Indoor monocular semantic scene completion (MSSC) is notably more challenging than its outdoor counterpart due to complex spatial layouts and severe occlusions. While transformers are well suited for modeling global dependencies, their high memory cost and difficulty in reconstructing fine-grained details have limited their use in indoor MSSC. To address these limitations, we introduce AdaSFormer, a serialized transformer framework tailored for indoor MSSC. Our model features three key designs: (1) an Adaptive Serialized Transformer with learnable shifts that dynamically adjust receptive fields; (2) a Center-Relative Positional Encoding that captures spatial information richness; and (3) a Convolution-Modulated Layer Normalization that bridges heterogeneous representations between convolutional and transformer features. Extensive experiments on NYUv2 and Occ-ScanNet demonstrate that AdaSFormer achieves state-of-the-art performance. The code is publicly available at: https://github.com/alanWXZ/AdaSFormer.

Figures & tables

Figure 2: A d a S F o r m e r Framework . (a) Overall network architecture: It mainly consists of a 2D encoder, a depth estimation network, a 2D-to-3D projection module, and a 3D network. (b) Adaptive Serialized Transformer (ASF): This is the core of our method. A set of ASF blocks forms the encoder of the 3D network. The 3D voxels are adaptively converted into 1D patches, which are then augmented with center-relative positional encoding. These patches are processed through standard self-attention and feed-forward layers. Through the proposed Convolution-Modulated Layer Normalization (CMLN), the heterogeneous features from the transformer and convolutional layers are bridged. The resulting features are then fed into a DDR block for scene completion. (c) Convolution-Modulated Layer Normalization (CMLN): Modulates features to integrate transformer and convolutional representations effectively. (d) Center-Relative Positional Encoding: Enhances the richness of input information by modeling the relative spatial relationships of patches.

Figure 3: (a) illustrates the Z-order serialization of a 2D image, as well as that of a sparse 2D image, where the letters “C” and “V” in the figure indicate the locations of objects in the spatial domain. (b)-(e) illustration of Z-order serialization with shifts of 0, 3, 6, and 9 (from left to right). Different shift settings significantly affect the coverage and overlap between adjacent patches, especially for large patch sizes.

Method	Venue	SC IoU (%)	$\blacksquare$ Ceiling	$\blacksquare$ Floor	$\blacksquare$ Wall	$\blacksquare$ window	$\blacksquare$ chair	$\blacksquare$ bed	$\blacksquare$ sofa	$\blacksquare$ table	$\blacksquare$ tvs	$\blacksquare$ furniture	$\blacksquare$ objects	SSC mIoU (%)
LMSCNet † [ 60 ]	3DV’20	$33.93$	$4.49$	$88.41$	$4.63$	$0.25$	$3.94$	$32.03$	$15.44$	$6.57$	$0.02$	$14.51$	$4.39$	$15.88$
AICNet † [ 33 ]	CVPR’20	$30.03$	$7.58$	$82.97$	$9.15$	$0.05$	$6.93$	$35.87$	$22.92$	$11.11$	$0.71$	$15.90$	$6.45$	$18.15$
3DSketch † [ 8 ]	CVPR’20	$38.64$	$8.53$	$90.45$	$9.94$	$5.67$	$10.64$	$42.29$	$29.21$	$13.88$	$9.38$	$23.83$	$8.19$	$22.91$
MonoScene [ 4 ]	CVPR’22	$42.51$	$8.89$	$93.50$	$12.06$	$12.57$	$13.72$	$48.19$	$36.11$	$15.13$	$15.22$	$27.96$	$12.94$	$26.94$
NDC-Scene [ 90 ]	ICCV’23	$44.17$	$12.02$	$93.51$	$13.11$	$13.77$	$15.83$	$49.57$	$39.87$	$17.17$	$24.57$	$31.00$	$14.96$	$29.03$
ISO [ 91 ]	ECCV’24	$47.11$	$14.21$	$93.47$	$15.89$	$15.14$	$18.35$	$50.01$	$40.82$	$18.25$	$25.90$	$34.08$	$17.67$	$31.25$

Table 1: Comparisons among state-of-the-art MSSC methods on the NYUv2 dataset [ 63 ] . The symbol

{\dagger}

denotes the results presented by [ 4 ] . The Best and 2nd Best scores from each metric are highlighted in Orange and Teal , respectively.

Method	Venue	IoU (%)	$\blacksquare$ ceiling	$\blacksquare$ floor	$\blacksquare$ wall	$\blacksquare$ window	$\blacksquare$ chair	$\blacksquare$ bed	$\blacksquare$ sofa	$\blacksquare$ table	$\blacksquare$ tvs	$\blacksquare$ furniture	$\blacksquare$ objects	mIoU (%)
TPVFormer [ 23 ]	CVPR’23	$33.39$	$6.96$	$32.97$	$14.41$	$9.10$	$24.01$	$41.49$	$45.44$	$28.61$	$10.66$	$35.37$	$25.31$	$24.94$
GaussianFormer [ 24 ]	ECCV’24	$40.91$	$20.70$	$42.00$	$23.40$	$17.40$	$27.0$	$44.30$	$44.80$	$32.70$	$15.30$	$36.70$	$25.00$	$29.93$
MonoScene [ 4 ]	CVPR’22	$41.60$	$15.17$	$44.71$	$22.41$	$12.55$	$26.11$	$27.03$	$35.91$	$28.32$	$6.57$	$32.16$	$19.84$	$24.62$
ISO [ 91 ]	ECCV’24	$42.16$	$19.88$	$41.88$	$22.37$	$16.98$	$29.09$	$42.43$	$42.00$	$29.60$	$10.62$	$36.36$	$24.61$	$28.71$
SurroundOCC [ 76 ]	ICCV’23	$42.52$	$18.90$	$49.30$	$24.80$	$18.00$	$26.80$	$42.00$	$44.10$	$32.90$	$18.60$	$36.80$	$26.90$	$30.83$
A d a S F o r m e r	Ours	$54.63$	$37.66$	$57.34$	$40.51$	$29.49$	$43.29$	$60.41$	$63.08$	$47.05$	$29.63$	$54.08$	$36.15$	$45.33$

Table 2: Comparisons among state-of-the-art MSSC methods on the Occ-ScanNet dataset [ 91 ] . The symbol

{\dagger}

denotes the results presented by [ 4 ] . The Best and 2nd Best scores from each metric are highlighted in Orange and Teal , respectively. Symbol * denotes the results obtained using their official code trained on the Occ-ScanNet dataset.

Table 5

Figure 4: Visualization of the ablation results on the Occ-ScanNet dataset [ 91 ] . Introducing AdaSFormer significantly improves the model’s capability to capture spatial and semantic relationships, leading to more complete and coherent object completion.

Figure 5: Qualitative Comparisons of semantic scene completion results on the NYUv2 testset [ 63 ] with different methods. Figures from left to right: (a) The single RGB input; (b) The ground truth; (c) ISO [ 91 ] ; (d) Baseline; and (e) Our proposed method.

Method	ISO [ 91 ]	AdaSFormer	w/o ADA	w/o CRPE	w/o CMLN
Latency	$535.2$	$178.9$	$\mathbf{156.0}$	$177.4$	$174.3$

Table 6: Latency (in

\mathrm{ms}

) study on each component.

3D Scene Completion

3D scene completion infers missing geometry, occupancy, semantics, and appearance from partial images, point clouds, or depth scans.

See all 41 papers

CrowdOcc: Monocular Semantic Scene Completion for Quadruped Robots in Crowded Indoor Environments
Oct 2, 2026Feiyang Chen, Jincheng Hu, Yiduo Chen +53D Occupancy Prediction 3D Scene Completion
Tongji University, Shanghai, China. · Loughborough University, Loughborough, United Kingdom.
Exploring Easy Boosts for Lidar Semantic Scene Completion
Jun 2, 2026Tetiana Martyniuk, Jonathan Seele, Alexandre Boulch +33D Semantic Segmentation Pseudo-Labeling
Inria, France · ETH Zurich, Switzerland · valeo.ai, France +1
Adaptive Multi-Scale Channel-Spatial Attention Aggregation Framework for 3D Indoor Semantic Scene Completion Toward Assisting Visually Impaired
Feb 18, 2026Qi He, XiangXiang Wang, Jingtao Zhang +5Visually Impaired Navigation Assistance 3D Scene Completion
School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China. · Sichuan Provincial Key Laboratory for Human Disease Gene Study, Sichuan Academy of Medical Sciences & Sichuan Provincial People’s Hospital, University of Electronic Science and Technology of China, Chengdu, Sichuan, China. · Tibetan Language Intelligence National Key Laboratory, Qinghai Normal University, Xining, Qinghai, China. +2

Oct 2, 2026J/K move · Enter open · S save

CrowdOcc: Monocular Semantic Scene Completion for Quadruped Robots in Crowded Indoor Environments

Feiyang Chen, Jincheng Hu, Yiduo Chen, Jihao Li+4

Tongji University, Shanghai, China. · Loughborough University, Loughborough, United Kingdom.

3D Occupancy Prediction 3D Scene Completion Robotic Perception

Open paper AskPDF arXiv

Monocular semantic scene completion (SSC) for quadruped robots remains underexplored in real crowded indoor environments, where human-scene occlusion disrupts static geometry and human occupancy predictions are often incomplete or spatially misplaced. We present CrowdOcc, an RGB-D dataset and monocular SSC framework for this setting. CrowdOcc contains 25.1K frames from 11 indoor scenes, with semantic occupancy annotations constructed through static dynamic decoupling. Our framework combines: (i) Normal Guided Scene Geometry Fusion (NGSGF) to complement depth-aware lifting with surface-normal cues for occlusion robust geometry; and (ii) Human-Centric Sparse Interaction (HCSI) to selectively model human-human and local human scene relations in 3D. Our method achieves state-of-the-art SSC performance on CrowdOcc's scene-disjoint test set, reaching 15.80 IoU, 11.40 mIoU, and 46.23 Human IoU, demonstrating generalization to unseen indoor scenes.

Abstract

3D Scene Completion

Explore similar work

CrowdOcc: Monocular Semantic Scene Completion for Quadruped Robots in Crowded Indoor Environments

Exploring Easy Boosts for Lidar Semantic Scene Completion

Adaptive Multi-Scale Channel-Spatial Attention Aggregation Framework for 3D Indoor Semantic Scene Completion Toward Assisting Visually Impaired