ToLL: Topological Layout Learning with Asymmetric Cross-View Structural Distillation for 3D Scene Graph Generation Pretraining

Yucheng Huang; Luping Ji; Xiangwei Jiang; Wen Li; Mao Ye

cs.CVMar 30, 2026

ToLL: Topological Layout Learning with Asymmetric Cross-View Structural Distillation for 3D Scene Graph Generation Pretraining

Authors: Yucheng Huang, Luping Ji, Xiangwei Jiang, Wen Li, Mao Ye

Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China

Abstract

3D Scene Graph (3DSG) generation plays a pivotal role in spatial understanding and affordance perception. To mitigate generalization issues from data scarcity, joint-embedding and generative proxy tasks are proposed to pre-train 3DSG representations on predicate label-free datasets. Currently, generative pre-training usually bypasses the semantic corruption caused by the geometric augmentations in joint-embedding, but cannot avoid a negative problem ``Geometric Shortcut." In this problem, exposing dense object spatial and scale priors will induce models to trivially reconstruct scenes by interpolating object positions, rather than learning the underlying topological constraints provided by edges. To address this issue, we propose a Topological Layout Learning (ToLL) for 3DSG generation pretraining framework. In detail, we design an Anchor-Conditioned Topological Geometry Reasoning. It adopts a recurrent GNN to recover the global layout of zero-centered subgraphs (the non-visible spatial features) by one anchor with sparse spatial prior. Considering the absence of spatial layout information within the objects, it creates an information bottleneck, compelling our model to recover the full scene layout by leveraging predicate representation learning. Moreover, we construct a Structural Multi-view Augmentation to avoid semantic corruption, enhancing 3DSG representations via self-distillation. The extensive experiments on special dataset demonstrate that our ToLL could often improve 3DSG pertaining quality, outperforming state-of-the-art baselines.

Figures & tables

Panel of Figure 1 — Fig. 1: Proxy task scheme comparison of 3DSG generation: (a) cross-modal Vision-Language representation, (b) trivial scene generation, (c) non-trivial scene generation, $i.e.$ , our ToLL.

Panel of Figure 2 — Fig. 2: The motivation of designing Anchor-Conditioned Topological Geometric Reasoning (ACTGR). We observe that exposing complete object geometries (shape, scale, and location) will trigger shortcut learning, causing edge encoder and relative message gradients (plotted by red curves) drop sharply. In contrast, ACTGR can effectively avoid this problem (refer to the blue curves).

Panel of Figure 3 — Fig. 3: Our 3D Scene Graph Pretraining scheme via Topological Layout Learning with Structural Multi-view Augmentation.

Role	View ID	Source Input	Point M-Ratio	Edge M-Ratio	Objective
Student	$S_{v1}$	Origin	0.8	0.2	Prediction / Gradient Update
	$S_{v2}$	Augmented	0.8	0.6
	$S_{v3}$	Origin	0.8	0.6
Teacher	$T_{v4}$	Origin	0.2	0.2	Target Generation (EMA)
Teacher	$T_{v5}$	Augmented	0.1	0.1	Target Generation (EMA)

TABLE I: View settings. Source Input indicates whether the input points are from the original data or augmented ones. M-Ratio denotes the masking ratio. The ACTGR strategy is exclusively implemented on the

S_{v1}

view.

Baselines	Pre-training Method	Object		Predicate				Triplet		SGCLs		PredCLs
Baselines	Pre-training Method	A@1	A@5	A@1	A@3	mA@1	mA@3	mA@50	mA@100	mR@20	mR@50	mR@20	mR@50
SGPN pn [ 3 ]	$\times$	50.32	74.56	89.89	98.15	40.63	63.41	52.74	65.58	19.7	22.6	32.1	38.4
SGFN pn [ 10 ]	$\times$	53.67	77.18	90.19	98.17	41.89	70.82	58.37	67.61	20.5	23.1	46.1	54.8
SGFN pt	$\times$	56.04	79.37	89.22	97.65	46.69	71.84	60.05	70.26	29.3	30.8	55.7	62.1
VL-SAT pn [ 7 ]	$\times$	55.66	78.66	89.81	98.45	54.03	77.67	65.09	73.59	31.8	32.4	57.8	64.2
VL-SAT pt	$\times$	57.84	78.69	89.76	98.13	52.43	73.35	63.55	72.48	30.3	32.8	54.7	63.8

TABLE II: Comparisons with state-of-the-arts on the 3DSSG dataset. The inference model backbones are denoted by symbols: “ pn ” for PointNet and “ pt ” for PointTransformer. “ PointDif ”: Weights of object encoder initialized with PointDif [ 40 ] . “ ToLL ” or “ ToLL++ ”: Weights of complete 3DSG encoders initialized with our ToLL. “ +MLP only ”: Only MLP heads are fine-tuned.

Method	Pre-training Method	Predicate Classification Metrics			Triplet Classification Metrics
Method	Pre-training Method	Head (mA@3 / mA@5)	Body (mA@3 / mA@5)	Tail (mA@3 / mA@5)	Unseen (A@50 / A@100)	Seen (A@50 / A@100)
SGFN pn [ 10 ]	$\times$	95.08 / 99.38	70.02 / 87.81	38.67 / 58.21	22.59 / 35.68	71.44 / 80.11
SGFN pt	$\times$	96.18 / 99.46	76.74 / 87.41	48.25 / 62.73	25.54 / 46.69	72.87 / 80.08
VL-SAT pn [ 7 ]	$\times$	96.31 / 99.21	80.03 / 93.64	52.38 / 66.13	31.28 / 47.26	75.09 / 82.25
VL-SAT pt	$\times$	95.97 / 98.92	78.75 / 88.84	54.44 / 68.90	32.87 / 46.32	77.28 / 83.37
CCL-3DSG pn [ 8 ]	$\times$	98.54 / 99.78	84.72 / 96.03	61.24 / 75.91	36.72 / 52.47	80.58 / 88.92

TABLE III: Performance on Predicate Classification (Head, Body, Tail) and Triplet (Zero-Shot setting). “ pn ” for PointNet and “ pt ” for PointTransformer. “ ToLL ” or or “ ToLL++ ”: Weights of complete 3DSG backbone initialized with our ToLL.

Fig. 4: Predicate A@1 for all predicate categories.

Panel of Figure 8 — Fig. 5: Predicate A@1 for all predicate categories.

Method (SGFN)/ Settings	Obj	Pred		Triplet
Method (SGFN)/ Settings	A@1	mA@1	mA@3	mA@50	mA@100
(1) Anchor-Conditioned Topological Geometric Reasoning, ACTGR
Baseline Global Layout	57.62	47.28	70.20	60.39	70.85
ACTGR	58.46	53.87	75.69	64.92	73.66
(2) Structural Multi-view Augmentation, SMA
SMA w/ ACTGR	58.68	54.59	81.36	66.58	74.32

TABLE IV: Ablation Studies on 3DSSG. We analyze the impact of Anchor-Conditioned Topological Geometric Reasoning (ACTGR), Structural Multi-view Augmentation (SMA), initialization strategies, and Decoupled Layout Recovery.

Method	Visible Anchors	Object A@1	Predicate mA@1	Triplet mA@50
ACTGR Only	Single Anchor	58.46	53.87	64.92
	50% Visible Anchors	57.83	46.80	61.30
	100% Visible Anchors	57.62	47.28	60.39
ToLL (Ours)	Single Anchor	58.68	54.59	66.58
	50% Visible Anchors	58.96	49.85	62.20
	100% Visible Anchors	58.49	47.76	60.65

TABLE V: Ablation study of different anchor ratios.

Panel of Figure 11 — Fig. 6: Visualization of latent features clustering.

Panel of Figure 12 — Fig. 7: Visualization of clustering confusion matrix.

Panel of Figure 13 — Fig. 8: Analysis of accuracy curves (Train and Validation).

Panel of Figure 14 — Fig. 9: Visualization of diffusion layout recovery.

3D SGG

3D scene graph generation (3D SGG) represents objects, parts, and their spatial or functional relations as structured models of 3D environments.

See all 65 papers

GHOST: Geometry-Constrained Hybrid Diffusion for Open-Vocabulary 3D Scene Graph Generation
Oct 12, 2026Changqun Feng, Wangxiandi Yin, Xin Hu +43D SGG 3D Scene Understanding
University of Electronic Science and Technology of China Chengdu, Sichuan, China · Jilin University Changchun, Jilin, China
DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors
Jul 1, 2026Seok-Young Kim, Abdelrahman Elskhawy, Taewook Ha +4Scene Graph Generation 3D SGG
Korea Advanced Institute of Science and Technology · Technical University of Munich · Munich Center for Machine Learning (MCML) +1
SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation
Jun 13, 2026Mengshi Qi, Changsheng Lv, Zijian Fu +2Class-Incremental Learning 3D SGG
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, China

Oct 12, 2026J/K move · Enter open · S save

GHOST: Geometry-Constrained Hybrid Diffusion for Open-Vocabulary 3D Scene Graph Generation

Changqun Feng, Wangxiandi Yin, Xin Hu, Ruiting Dai+3

University of Electronic Science and Technology of China Chengdu, Sichuan, China · Jilin University Changchun, Jilin, China

3D SGG 3D Scene Understanding 3D Spatial Reasoning 3D Diffusion Models

Open paper AskPDF arXiv

3D scene graph generation requires understanding both object semantics and metric spatial relations. However, existing open-vocabulary methods often struggle with spatial predicates because language similarity reflects semantic co-occurrence rather than explicit geometry. We propose GHOST, a Geometry-constrained Hybrid diffusion framework for Open-vocabulary 3D Scene graph generaTion. GHOST jointly models continuous node geometry and discrete edge predicates, following a geometry-to-constraint-to-predicate denoising process to ensure geometrically consistent relation prediction. It introduces language-grounded constraint induction, which maps predicate embeddings to differentiable geometric constraints that guide both layout refinement and predicate reweighting. A Geometry-Topology Collaborative Denoiser further couples node and edge reasoning through noise-adaptive interaction and orientation-aware modeling. Experiments on 3DSSG demonstrate state-of-the-art performance in both closed-set and open-vocabulary settings, including over 17-point gains over RelationField on spatial predicates.

Abstract

3D SGG

Explore similar work

GHOST: Geometry-Constrained Hybrid Diffusion for Open-Vocabulary 3D Scene Graph Generation

DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors

SGFormer++: Semantic Graph Transformer for Incremental 3D Scene Graph Generation