PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstruction

Xiang Zhang; Sohyun Yoo; Hongrui Wu; Chuan Li; Jianwen Xie; Zhuowen Tu

cs.CVMar 6, 2026

PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstruction

Authors: Xiang Zhang, Sohyun Yoo, Hongrui Wu, Chuan Li, Jianwen Xie, Zhuowen Tu

Organizations: UC San Diego · Lambda, Inc.

Abstract

We introduce PixARMesh, a method to autoregressively reconstruct complete 3D indoor scene meshes directly from a single RGB image. Unlike prior methods that rely on implicit signed distance fields and post-hoc layout optimization, PixARMesh jointly predicts object layout and geometry within a unified model, producing coherent and artist-ready meshes in a single forward pass. Building on recent advances in mesh generative models, we augment a point-cloud encoder with pixel-aligned image features and global scene context via cross-attention, enabling accurate spatial reasoning from a single image. Scenes are generated autoregressively from a unified token stream containing context, pose, and mesh, yielding compact meshes with high-fidelity geometry. Experiments on synthetic and real-world datasets show that PixARMesh achieves state-of-the-art reconstruction quality while producing lightweight, high-quality meshes ready for downstream applications.

Figures & tables

Figure 1: Comparison of PixARMesh with recent compositional scene reconstruction methods. PixARMesh predicts object poses and reconstructs native meshes in a single autoregressive decoding process, without relying on SDF-based surface extraction or layout optimization, producing compact and artist-ready mesh outputs.

Figure 2: Pipeline overview. Given an RGB image, we use pretrained models to extract the depth point cloud and image features for both the target object $i$ and the global scene. These local and global cues are fed into the Pixel-Aligned PC-Encoder to produce the fused latent code, which is then aggregated into a single latent vector via cross-attention. This latent vector conditions the Transformer Decoder, which predicts the object’s pose followed by its mesh token sequence.

SDF-based
Method	Scene-level			Object-level
Method	CD ( $\times 10^{-3}$ , $\downarrow$ )	CD-S ( $\times 10^{-3}$ , $\downarrow$ )	F-Score ( $\%$ , $\uparrow$ )	CD ( $\times 10^{-3}$ , $\downarrow$ )	F-Score ( $\%$ , $\uparrow$ )
InstPIFu [ 27 ]	213.4	124.9	13.72	44.74	29.63
Uni-3D [ 47 ]	218.3	113.3	12.99	—	—
Gen3DSR [ 13 ]	222.4	137.5	13.52	9.74	31.42
DeepPriorAssembly [ 51 ]	191.8	76.2	16.72	20.13	27.83

Table 1: Qualitative comparison with state-of-the-art methods on the 3D-FRONT [ 14 ] dataset. Following DepR [ 49 ] and DeepPriorAssembly [ 51 ] , we report object- and scene-level Chamfer Distance (CD; lower is better) and F-Score (higher is better). We additionally include the single-direction Chamfer Distance (CD-S) to account for missing instances.

Figure 3: Qualitative comparisons on the 3D-FRONT [ 14 ] dataset. For PixARMesh, we also show the mesh wireframe to highlight geometric quality.

Figure 4: Qualitative results on real images from Pix3D [ 40 ] , Matterport3D [ 3 ] , and ScanNet [ 10 ] datasets.

Method	Scene-level			Object-level
	CD	CD-S	F-Score	CD	F-Score
	( $\times 10^{-3}$ , $\downarrow$ )	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )
Two-stage	99.8	50.6	33.32	4.75	80.85
EdgeRunner-FT	119.8	48.0	27.81	4.75	80.57
PixARMesh (Ours)	98.8	49.1	33.55	4.04	82.27

Table 2: Ablation study on joint pose-mesh modeling.

Img Feat	Ctx Agg	Scene-level			Object-level
		CD	CD-S	F-Score	CD	F-Score
		( $\times 10^{-3}$ , $\downarrow$ )	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )
		57.78	23.75	41.02	5.29	77.22
	✓	55.44	27.16	42.84	5.56	78.14
✓		39.30	16.12	44.67	3.64	84.64
✓	✓	39.88	18.52	46.15	4.04	82.27

Table 3: Ablation studies on our point-cloud encoder design. Img Feat , Ctx Agg denote pixel-aligned image features and scene context aggregation, respectively.

GT Depth	GT Layout	CD	F-Score
GT Depth	GT Layout	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )
		4.13	81.64
✓		3.04	86.66
	✓	4.04	82.27
✓	✓	2.93	87.19

Table 4: Effects of depth and layout in object-level metrics.

GT Inputs			CD	CD-S	F-Score
Depth	Segm	Layout	( $\times 10^{-3}$ , $\downarrow$ )	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )
			98.84	49.08	33.55
✓			95.69	48.18	39.04
	✓		39.88	18.52	46.15
	✓	✓	16.30	4.40	63.91
✓	✓		25.07	10.05	59.63

Table 5: Effects of upstream (depth, segmentation, and layout) errors in scene-level metrics. Note that ground-truth layout implies ground-truth segmentation.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1 : Additional qualitative results on real images from Pix3D [ 40 ] , Matterport3D [ 3 ] and ScanNet [ 10 ] .

Method	$\|F\|$	$\|V\|$
InstPIFu [ 27 ]	1,942,029	970,951
Uni-3D [ 47 ]	141,130	70,844
BUOL [ 8 ]	55,493	27,750
Gen3DSR [ 13 ]	364,285	216,797
DeepPriorAssembly [ 51 ]	250,875	125,420
MIDI [ 19 ]	1,936,248	967,886

Appendix

Table A.1 : Face count

|F|

and vertex count

|V|

comparisons.

Method	GT Depth	Box IoU ( $\%$ )
MIDI [ 19 ]	$-$	31.07
DepR [ 49 ]		33.98
DepR [ 49 ]	$\checkmark$	36.67
PixARMesh-EdgeRunner (Ours)		56.76
PixARMesh-EdgeRunner (Ours)	$\checkmark$	70.37
PixARMesh-BPT (Ours)		54.54

Appendix

Table A.2 : Layout accuracy comparisons.

Method	Runtime
InstPIFu [ 27 ]	$19.8\text{\,}\mathrm{s}$
Uni-3D [ 47 ]	$3.1\text{\,}\mathrm{s}$
BUOL [ 8 ]	$6.5\text{\,}\mathrm{s}$
Gen3DSR [ 13 ]	$15.5\text{\,}\mathrm{min}$
DeepPriorAssembly [ 51 ]	$6.7\text{\,}\mathrm{min}$
MIDI [ 19 ]	$1.6\text{\,}\mathrm{min}$

Appendix

Table A.3 : Inference runtime comparisons.

Img Feat	Ctx Agg	Scene-level			Object-level
		CD	CD-S	F-Score	CD	F-Score
		( $\times 10^{-3}$ , $\downarrow$ )	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )	( $\times 10^{-3}$ , $\downarrow$ )	( $\%$ , $\uparrow$ )
		68.36	25.43	35.96	5.34	76.54
	✓	56.80	23.49	37.98	5.85	76.31
✓		57.03	23.70	41.51	4.25	81.25
✓	✓	49.82	22.06	42.18	4.57	80.30

Appendix

Table A.4 : Ablation studies on our point-cloud encoder design with BPT-based model. Img Feat , Ctx Agg denote pixel-aligned image features and scene context aggregation, respectively.

Autoregressive 3D Scene Generation

See all 14 papers

WorldRecon: Generative Multi-View Mesh Reconstruction for Large-Scale 3D World
Oct 13, 2026Mengchen Zhang, Yang Li, Chunshi Wang +23D Reconstruction 3D Mesh Reconstruction
Zhejiang University · Tencent Hunyuan · Fudan University
SceneReGen: Generative Reconstruction of 3D Scenes from a Single Image
Aug 25, 2026Zefan Tian, Yuteng Ye, Yiheng Zhang +53D Reconstruction Single-View 3D Reconstruction
Huawei · Northwestern Polytechnical University
GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction
May 22, 2026Katharina Schmid, Nicolas von Lützow, Jozef Hladký +23D Reconstruction Multi-View 3D Reconstruction
Technical University of Munich · Computing Systems Lab, Huawei Technologies, Switzerland

Oct 13, 2026J/K move · Enter open · S save

WorldRecon: Generative Multi-View Mesh Reconstruction for Large-Scale 3D World

Mengchen Zhang, Yang Li, Chunshi Wang, Tong Wu+1

Zhejiang University · Tencent Hunyuan · Fudan University

3D Reconstruction 3D Mesh Reconstruction Multi-View 3D Reconstruction Large-Scale 3D Reconstruction Indoor Scene Reconstruction 3D Diffusion Models

Open paper AskPDF arXiv

Reconstructing complete, high-quality meshes of large-scale scenes from unposed multi-view images requires faithful geometric grounding while plausibly completing surfaces that are unobserved or only sparsely captured. We present WorldRecon, a generative multi-view reconstruction framework for recovering complete, large-scale 3D scene meshes from unposed RGB images. WorldRecon uses a coarse, noisy global point cloud and camera poses estimated from the input images by a reconstruction model as a global scaffold for whole-scene generation and partitions it into overlapping spatial blocks. To robustly leverage these imperfect geometric priors, we equip the native 3D generative model TRELLIS.2 with a dedicated reconstruction-based conditioning architecture that generates each block from its local point-cloud crop and block-relevant image observations, producing sparse structure, detailed geometry, and PBR materials. To maintain scene-level coherence, we introduce neighbour-aware latent inpainting and a causal world-generation procedure that propagates structural, geometric, and material context across overlapping blocks. Together, these designs scale multi-view generative reconstruction to large-scale scenes, yielding coherent, unified meshes. Extensive experiments demonstrate that WorldRecon achieves state-of-the-art reconstruction performance, reducing Chamfer distance by 14.9% and 43.1% while improving F-score by 17.3% and 12.0% over the strongest baselines on synthetic and real-world indoor scenes, respectively, thereby demonstrating faithful and complete generative reconstruction as well as strong sim-to-real generalization. Project page: https://kszpxxzmc.github.io/WorldRecon-website/.

Abstract

Autoregressive 3D Scene Generation

Explore similar work

WorldRecon: Generative Multi-View Mesh Reconstruction for Large-Scale 3D World

SceneReGen: Generative Reconstruction of 3D Scenes from a Single Image

GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction