PAGER: Partial-to-global Alignment via Geometric and Relational Distillation

Akira-Miranda Adeyomi Adeniran-Lowe; Binod Singh; Lars Arnold Dethlefsen; Lazaros Nalpantidis; Theodora Kontogianni

cs.CVOct 1, 2026

PAGER: Partial-to-global Alignment via Geometric and Relational Distillation

Authors: Akira-Miranda Adeyomi Adeniran-Lowe, Binod Singh, Lars Arnold Dethlefsen, Lazaros Nalpantidis, Theodora Kontogianni

Organizations: Technical University of Denmark · Pioneer Center for Artificial Intelligence

Abstract

Pretrained 3D encoders are typically developed on globally reconstructed scenes expressed in a consistent world coordinate frame, whereas embodied systems must reason from partial, viewpoint-dependent observations in camera coordinates. We show that this shift from globally learned 3D feature spaces to realistic partial observations exposes a severe representation mismatch, which we find consistently across representative state-of-the-art encoders, including Sonata and Concerto. A frozen Sonata encoder with a global linear probe achieves 72.47 mIoU on full ScanNet scenes, but 2.57 mIoU on single-frame camera-coordinate inputs. Training-free gravity alignment recovers performance to 41.64 mIoU, showing that coordinate-frame mismatch is a dominant source of degradation but cannot be fully resolved through canonicalization alone. We introduce PAGER, a label-free adaptation method that aligns partial-view features with a frozen global 3D semantic space using only paired partial/global geometry. It learns lightweight adaptation modules while keeping the pretrained encoder and global segmentation probe frozen. Matched-point feature alignment anchors partial features to their global counterparts, while relational supervision preserves their similarity structure with respect to the global representation. Global geometry provides supervision only during training. Inference operates directly on the partial observation. Without partial-view labels, PAGER outperforms label-supervised PEFT on both Sonata and Concerto, and in zero-shot ScanNet $\rightarrow$ ScanNet++ transfer surpasses fully fine-tuned Sonata ( $53.93$ vs.\ $48.09$ mIoU), suggesting that preserving the frozen global representation can improve cross-dataset transfer.

Figures & tables

Figure 1: Problem formulation - Partial Semantic Segmentation. Given a 3D point cloud obtained from monocular 3D reconstruction with limited field-of-view (FoV)(left), our goal is to predict semantic labels for the visible points (top, circle) without relying on additional scene context. We propose PAGER , a framework specifically designed to address this challenging scenario. This setting mirrors real-world challenges in semantic scene understanding from monocular onboard cameras, where models must infer object semantics from partial and incomplete observations, for instance, identifying a bed region in a bedroom despite limited FoV and strong visual similarity to surrounding surfaces ( white, flat ). Here, the globally trained model Wu et al. (2025) misclassifies a bed as wall . Transparent points denote unobserved regions shown only for reference.

Figure 2: Two sources of degradation. Left: progressive geometric reduction (Global $\to$ Room Slice $\to$ View Crop $\to$ Single Frame) in world coordinates lowers performance due to missing context. Right: the same inputs with camera coordinates causes a larger drop, visible in both segmentation predictions (Tab. 1 ) and PCA feature visualizations (bottom row). Coordinate-frame mismatch is the dominant factor, geometry reduction further compounds the gap. Coordinate system origin shown by black/red arrows.

Panel of Table 1 — Table 1: Representation gap on ScanNet. A frozen Sonata encoder with a global linear probe degrades under geometric reduction in world coordinates ( $-5$ mIoU), and significantly under camera-coordinate shift ( $-70$ mIoU). Training-free canonicalization recovers part, but not all, of this loss.

Figure 3: PAGER architecture. Only the adaptation layers on the partial branch are trained. For matched points, an anchor $\mathcal{L}_{\text{cos}}$ (Eq. 2 ) and a relational loss $\mathcal{L}_{\text{KL}}$ (Eq. 4 ) are used. Encoder and linear probe remain frozen. At inference, only the partial branch is used.

Sonata ( Wu et al., 2025 )
				ScanNet			ScanNet++			ScanNet200
Method	Family	Superv.	Params	mIoU $\uparrow$	mAcc $\uparrow$	allAcc $\uparrow$	mIoU $\uparrow$	mAcc $\uparrow$	allAcc $\uparrow$	mIoU $\uparrow$	mAcc $\uparrow$	allAcc $\uparrow$
Frozen	frozen	none	–	$2.57$	$6.94$	$18.30$	$0.69$	$1.92$	$9.78$	$0.14$	$0.37$	$3.11$
Linear	fine-tune	sem.	$<$ 0.1M	$36.17$	$52.42$	$68.64$	$15.73$	$26.11$	$67.66$	$8.13$	$12.98$	$61.43$
Late blocks + linear	fine-tune	sem.	21.0M	$48.28$	$59.13$	$76.86$	$17.77$	$25.33$	$72.98$	$11.43$	$16.56$	$63.55$
Full + linear †	fine-tune	sem.	108.5M	$69.26$	$77.11$	$87.88$	$35.99$	$44.75$	$85.95$	$24.58$	$32.13$	$77.86$

Table 2: Semantic segmentation under partial-view adaptation. Results across two pretrained 3D backbones. Bold and underline denote best and second-best non-upper-bound results within each backbone.

Train: ScanNet-Partial (train) $\rightarrow$ Eval: ScanNet++-Partial (val)
Method	mIoU	mAcc	allAcc
Sonata ( Wu et al., 2025 ) (full + linear)	48.09	55.10	85.57
Sonata (+ linear)	26.41	35.10	72.32
Sonata (late blocks + linear)	34.15	42.64	78.78
Adapter ( Houlsby et al., 2019 )	30.69	47.95	74.93
GEM ( Tang et al., 2025 ) (+ linear)	46.10	56.17	84.19

Table 3: Cross-domain evaluation. Models are trained on ScanNet partial views and evaluated on ScanNet++ partial views. PAGER outperforms all baselines, including Sonata (full + linear) , indicating that stronger in-domain adaptation does not necessarily yield better cross-dataset transfer.

A1 Loss terms			A2 Key sampling			A3 Number of keys $M$
	ScanNet	ScanNet++		ScanNet	ScanNet++		ScanNet	ScanNet++
Local (Eq. 2 )	$63.31$	$51.50$	$k$ -NN keys	$49.71$	$35.76$	2048	$64.01$	$50.80$
Relational (Eq. 4 )	$55.58$	$44.73$	random	65.46	53.93	4096	65.46	53.93
Both	65.46	53.93				8192	$64.63$	$53.19$

Table 4: Ablation studies. mIoU on ScanNet Partial (in-domain) and ScanNet++ Partial (zero-shot cross-eval). All models are trained on ScanNet Partial. Defaults in bold.

M

values correspond to

1.4\%

2.8\%

, and

5.6\%

of global scene points.

Method	ScanNet-20 (val)		ScanNet-200 (val)
Method	Partial mIoU	2.5D mIoU	Partial mIoU	2.5D mIoU
D-DITR ( Knaebel et al., 2026 )	1.70	1.66	0.29	0.24
PAGER	(+63.76) 65.46	(+62.22) 63.88	(+18.96) 19.25	(+16.88) 17.12

Table 5: Evaluation with D-DITR. We evaluate both ScanNet Partial and its RGB-D reconstruction (ScanNet Partial-2.5D), using D-DITR ( Knaebel et al., 2026 ) . We report results using two publicly available D-DITR variants trained on ScanNet-20 and ScanNet-200 labels respectively.

Panel of Figure 4 — Figure 4: Qualitative results on ScanNet Partial. PAGER produces predictions closer to ground truth than the frozen Sonata baseline with a trained linear probe and label-supervised GEM, especially under severe FoV limitations and camera-coordinate inputs. Baselines often confuse visually similar planar surfaces: the mattress ( $1^{st}$ row ) is misclassified as toilet rather than bed, while the floor ( $3^{rd}$ row ) is mistaken for wall. PAGER better maintains object semantics through alignment to the global feature space and also produces plausible labels for unannotated points shown in black.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Pitch range	Frames	PAGER	GEM
Steep down ( $<-30^{\circ}$ )	5,567	$61.31$	$52.52$
Down ( $-30^{\circ}$ to $-10^{\circ}$ )	9,875	$65.82$	$60.89$
Level ( $-10^{\circ}$ to $10^{\circ}$ )	2,183	$61.44$	$52.45$
Up ( $10^{\circ}$ to $30^{\circ}$ )	166	$34.63$	$24.55$
Steep up ( $>30^{\circ}$ )	30	$18.38$	$18.95$

Appendix

Table 6: Partial-view segmentation by camera pitch. Frames are binned by pitch computed from each frame’s captured pose. The first three bins account for

98.9\%

of the evaluation set.

Panel of Figure 5 — Figure 5: In-the-wild results. Left: We use point clouds reconstructed from single RGB images with ( Wang et al., 2025a ) as input. Sonata (linear) and GEM mislabel flat regions as wall , while PAGER , recovers the door . Right: On stereo point clouds from a mobile robot in unseen environments, PAGER produces coherent segmentations despite the distribution shift, correctly recognizing the door .

	wall	floor	cabinet	bed	chair	sofa	table	door	window	bookshelf	picture	counter	desk	curtain	refrigerator	shower curtain	toilet	sink	bathtub	otherfurniture
Model
Sonata (full + linear)	80.52	94.47	60.73	81.46	86.28	79.75	75.21	59.03	52.28	70.75	32.08	59.79	65.80	68.85	55.96	60.83	94.18	70.28	89.73	47.14
Sonata (late blocks + linear)	66.05	84.67	43.32	64.19	68.22	62.79	62.21	33.72	29.78	50.01	18.52	36.21	44.87	37.95	34.81	38.94	67.46	42.73	61.04	18.03
Sonata (+linear)	62.15	83.24	28.38	45.68	52.16	42.67	49.71	12.33	25.71	36.28	24.85	27.43	29.09	33.64	19.74	34.36	44.71	30.75	30.79	9.68
GEM	76.82	93.19	51.45	72.95	79.54	72.86	66.44	51.06	49.59	64.47	36.47	53.66	49.72	57.94	48.31	48.48	74.34	49.39	64.25	35.32
PAGER	80.94	93.79	55.32	75.45	81.17	71.90	70.37	62.07	57.58	64.83	32.93	55.69	55.72	74.58	49.96	68.89	82.35	61.02	76.78	37.85

Appendix

Table 7: Per class IoU on ScanNet-Partial (val) for PAGER trained on ScanNet-Partial. We report IoU for 20 semantic classes of ScanNet.

	wall	floor	cabinet	bed	chair	sofa	table	door	window	bookshelf	picture	counter	desk	curtain	refrigerator	shower curtain	toilet	sink	bathtub	otherfurniture
Model
Sonata (full + linear)	82.84	93.72	48.28	59.02	85.62	69.63	59.73	45.26	41.06	33.85	29.68	53.54	0.00	66.97	35.09	0.00	78.61	46.20	0.00	32.82
Sonata (late blocks + linear)	75.57	87.73	31.41	34.75	70.10	54.37	48.25	28.07	28.13	25.90	14.00	29.95	0.00	27.31	22.98	0.00	54.22	34.95	0.00	15.24
Sonata (+linear)	68.12	78.55	19.28	16.33	46.71	35.30	43.07	7.75	22.44	12.23	23.55	21.32	0.00	33.92	9.34	0.00	40.71	30.98	0.00	18.54
GEM	82.00	91.44	51.69	39.25	73.95	64.86	58.21	40.98	48.08	43.46	36.25	44.84	0.00	63.86	26.84	0.00	74.34	54.59	0.00	27.29
PAGER	88.32	96.14	58.84	67.36	83.48	72.71	60.08	67.61	46.18	58.48	33.12	47.41	0.00	77.75	43.75	0.00	69.34	65.62	0.00	42.33

Appendix

Table 8: Per class cross evaluation (IoU) on ScanNet++-Partial (val). for PAGER trained on ScanNet-Partial for our cross-domain setup. We report IoU for 20 semantic classes of ScanNet.

	fireplace	light	decoration	clothes dryer	stairs	windowsill	hair dryer	tv stand	curtain	end table	backpack	paper cutter	dishwasher	coffee maker	coffee kettle	ceiling	shoe	scale	dish rack	blackboard
Model
Sonata (full + linear)	10.67	6.17	0.81	36.62	48.93	9.24	0.00	25.92	61.96	33.03	44.14	28.01	50.87	58.68	52.89	77.95	34.57	47.94	66.60	71.74
Sonata (late blocks + linear)	0.01	2.11	0.73	1.39	21.48	6.10	0.00	25.10	37.06	11.13	13.38	3.32	5.97	13.53	0.00	43.95	8.01	0.80	18.15	13.52
Sonata (+ linear)	0.00	1.11	3.95	0.03	8.61	9.92	0.00	14.21	30.50	0.88	19.72	10.60	0.35	12.50	5.31	49.95	23.46	0.00	2.76	7.63
GEM	3.54	8.19	9.25	5.99	37.74	9.52	1.17	33.01	60.00	16.27	44.24	27.62	25.64	34.30	28.82	73.73	44.89	34.21	50.37	62.84
PAGER	52.86	36.76	32.86	25.21	54.66	25.64	14.24	44.86	71.16	26.18	26.36	8.71	4.88	11.43	2.44	47.25	18.12	5.02	18.76	25.54

Appendix

Table 9: Per class IoU on ScanNet200-Partial (val). We report 20 representative semantic classes: 10 where PAGER outperforms GEM and 10 where GEM outperforms PAGER .

	coat hanger	blanket	refrigerator	bed	shower wall	towel	kitchen cabinet	picture	printer	poster	tap	book	whiteboard eraser	shoes	mouse	speaker	laptop	computer tower	keyboard	smoke detector
Model
Sonata (full + linear)	2.64	33.21	35.83	44.03	46.10	31.91	45.91	24.98	36.35	0.51	31.31	43.11	39.96	50.27	44.76	20.22	50.35	78.00	73.77	52.61
Sonata (late blocks + linear)	1.66	29.41	9.63	32.58	18.97	13.40	32.36	13.36	14.49	1.36	8.62	18.15	8.56	21.96	5.62	2.26	2.10	37.62	25.29	7.21
Sonata (+ linear)	3.02	37.07	2.02	20.71	4.87	13.70	16.31	19.78	9.97	5.01	16.28	23.45	7.34	16.25	12.39	3.51	8.60	21.74	34.20	17.15
GEM	3.83	37.24	30.36	39.57	32.70	32.61	45.10	32.70	25.31	3.47	36.06	38.71	28.70	47.72	35.80	33.99	42.90	64.61	65.13	47.56
PAGER	21.40	52.40	43.06	51.71	43.31	42.37	53.30	40.72	33.15	10.73	9.16	11.02	0.82	19.43	4.86	0.93	9.60	27.25	24.35	0.25

Appendix

Table 10: Per class IoU on ScanNet++-Partial (val). We report IoU for 20 out of 100 semantic classes for visualisation purposes. 10 where PAGER outperforms GEM and 10 where GEM outperforms PAGER

Model	Variant	mIoU	mAcc	AllAcc
Sonata	KL	65.46	80.50	85.93
Sonata	Sampled Gram	67.27	77.75	87.62
Concerto	KL	71.67	87.77	89.96
Concerto	Sampled Gram	74.14	85.26	90.66

Appendix

Table 11: Performance comparison across different models and variants. Results shown for Sonata and Concerto architectures using KL and Sampled Gram methods.

Method	mIoU	mAcc	allAcc
PTv3	1.30	5.03	14.28
Sonata (Decoder)	2.60	6.20	24.14

Appendix

Table 12: Globally trained PTv3 vs. Sonata (Decoder) on the partial setup.

Figure 6: PAGER losses visualization.

Component	Specification
Operating system	AlmaLinux release 9.6
CPU	AMD EPYC 9354 32-Core
RAM	768GB
GPU	NVIDIA H100 PCIe 80GB
CUDA version (system)	13.0
Python version	3.12.9

Appendix

Table 13: Hardware and software environment used for all experiments.

Optimization
Configuration	ScanNet-20/200 Partial	ScanNet++ Partial
Optimizer	AdamW	AdamW
Learning rate	$2\times 10^{-5}$	$2\times 10^{-5}$
LR schedule	Constant	Constant
Weight decay	$2\times 10^{-2}$	$2\times 10^{-2}$
Gradient clipping	3.0	3.0

Appendix

Table 14: PAGER training configuration for ScanNet Partial and ScanNet++ Partial.

Dataset	Src	Scenes			$\overline{\text{pts/scene}}$		Overall
		Train	Val	Test	Train	Val	Scenes	Points
ScanNet ( Dai et al., 2017 )	real	1.2K	312	100	145K	159K	1.6K	242M
ScanNet-Partial	real	19.6K	21.4K	N/A	6.5K	6.6K	40.9K	267M
ScanNet++ ( Yeshwanth et al., 2023 )	real	856	50	50	1.50M	1.51M	956	1.43B
ScanNet++-Partial	real	38.3K	7.1K	N/A	22.7K	21.6K	12.8K	276M

Appendix

Table 15: Dataset statistics. Number of scenes and average points per scene. Partial variants contain multiple partial reconstructions per scene therefore, we sample /100 for training on ScanNet and /60 for training on ScanNet++. Evaluation follows /30 in ScanNet and /60 in ScanNet++.

Explore similar work

Jul 1, 2026cs.CV

Robust 3D Alignment of Generative Reconstructions via Partial Monocular Observations

Aligning generative 3D reconstructions with partial monocular observations is a critical but under-explored challenge in computer vision. This task is inherently ill-posed due to severe asymmetries between noisy, sparse monocular inputs and dense generative priors, whose scale ambiguity and geometric hallucinations, combined with the lack of initial overlap, render traditional registration pipelines ineffective. To resolve these issues, we propose a training-free and interpretable geometric alignment framework that grounds generative 3D priors via a 3D similarity transformation (Sim(3)), which can recover accurate metric scale and pose. Specifically, we introduce an explicit scale factor to resolve metric ambiguity and employ a coarse-to-fine alignment strategy, leveraging geometry-aware descriptors for robust initialization and a decoupled closed-form solver for precision refinement. In addition, we introduce a Hallucination Filtering operation to effectively suppress outliers caused by hallucinated geometry. To evaluate alignment performance under these extreme conditions, we introduce GenPMOAlign--Where2Place, a rigorous benchmark specifically designed for Generative-to-Partial Monocular Observational Alignment. Experiments demonstrate that our method achieves stable and accurate registration, substantially outperforming both classical geometric pipelines and state-of-the-art learning-based baselines. Code and the benchmark will be publicly released.

Yuchen Zhang, Luanyuan Dai, Yiwei Wang +7

School of Computer Science, Shanghai Jiao Tong University, 200240, Shanghai, China · School of Computer Science and Engineering, Nanjing University of Science and Technology, 210094, Nanjing, China · School of Engineering and Applied Science, University of Pennsylvania, 19104, Philadelphia, USA +7

3D Reconstruction Monocular Feature Alignment

Ask PDF

May 26, 2026cs.CV

G3T Up! Gravity Aligned Coordinate Frames Simplify Pointmap Processing

Modern feed-forward 3D reconstruction methods like VGGT predict pixel-aligned pointmaps in camera-centric coordinate frames. However, this choice of coordinate frame is not always optimal. We propose instead to predict pointmaps in upright, gravity-aligned frames that exploit strong structural cues present in many real-world scenes. Unlike camera-centric frames, gravity-aligned frames share a common vertical axis across viewpoints, reducing the rotational degrees of freedom needed to relate pointmaps to one another. To this end, we introduce the Gravity Grounded Geometry Transformer (G3T), fine-tuned from existing models on gravity-aligned 3D data. G3T produces highly accurate gravity-aware predictions, including upright pointmaps and camera-to-gravity poses. We further introduce G3T-Long, a submap-based incremental 3D reconstruction pipeline that leverages the reduced rotational degrees of freedom afforded by upright frames to achieve significantly improved reconstruction accuracy.

Bharath Raj Nagoor Kani, Noah Snavely

Cornell University

Feed-Forward 3D Reconstruction 3D Generation Gravity

Ask PDF

Oct 5, 2026cs.CV

Less Context, Better Geometry: Masked Geometric Encoder for Robust 3D Foundation Models

Recent progress in 3D foundation models has enabled rapid 3D reconstruction and camera calibration by leveraging learned 3D priors from vast amount of spatial data. However, the all-to-all global attention design leads to quadratic complexity and limits long-sequence inference; unconstrained cross-view interactions also can propagate unreliable evidence from occluded or visually similar but geometrically distant views. In this paper, We introduce a Masked Geometric Encoder (MGE), which promotes the learning of robust geometric representations under incomplete cross-view context. During training, MGE strategically drops frame tokens from global attention and distills from a pretrained full-context teacher model. This allows the model to learn an intrinsically richer per-frame representation while providing sufficient intermediate supervision to avoid performance degradation. Through extensive experiments, we show that MGE leads to much stronger performance under occlusion and doppelganger views while retaining high performance on standard benchmarks. Such a richer frame representation also leads to more effective token reduction during inference. To this end, we develop a novel Anchor-Guided Adaptive token merging technique that preserves representative anchor frames while jointly merging redundant tokens from the remaining views. Compared to other efficient inference approaches, we can achieve inference speedup while consistently maintaining higher reconstruction quality, particularly in limited-view settings.

Zhimin Shao, Xijun Liu, Zhaoliang Zhang +4

Department of Electrical and Computer Engineering Johns Hopkins University · Department of Data Science University of Virginia

3D Foundation Models

Ask PDF