A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations

Neelu Madan; Àlex Pujol; Andreas Møgelmose; Sergio Escalera; Kamal Nasrollahi; Graham W. Taylor; Thomas B. Moeslund

cs.CVMar 14, 2026

A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations

Authors: Neelu Madan, Àlex Pujol, Andreas Møgelmose, Sergio Escalera, Kamal Nasrollahi, Graham W. Taylor, Thomas B. Moeslund

Organizations: Aalborg University & Pioneer Centre for AI, Denmark · Universitat de Barcelona & Computer Vision Center, Spain · Milestone Systems, Denmark · University of Guelph & Vector Institute, Canada

Abstract

Slot attention has emerged as a powerful framework for unsupervised object-centric learning, decomposing visual scenes into a small set of compact vector representations called slots, each capturing a distinct region or object. However, these slots are learned in Euclidean space, which provides no geometric inductive bias for the hierarchical relationships that naturally structure visual scenes. In this work, we propose a simple post-hoc pipeline to project Euclidean slot embeddings onto the Lorentz hyperboloid of hyperbolic space, without modifying the underlying training pipeline. We construct five-level visual hierarchies directly from slot attention masks and analyse whether hyperbolic geometry reveals latent hierarchical structure that remains invisible in Euclidean space. Integrating our pipeline with SPOT (images), VideoSAUR (video), and SlotContrast (video), We find that hyperbolic projection exposes a consistent scene-level to object-level organisation, where coarse slots occupy greater manifold depth than fine slots, which is absent in Euclidean space. We further identify a "curvature--task tradeoff": low curvature ( $c{=}0.2$ ) matches or outperforms Euclidean on parent slot retrieval, while moderate curvature ( $c{=}0.5$ ) achieves better inter-level separation. Together, these findings suggest that slot representations already encode latent hierarchy that hyperbolic geometry reveals, motivating end-to-end hyperbolic training as a natural next step. Code and models are available at github.com/NeeluMadan/HHS.

Figures & tables

Figure 1: Visual hierarchy on the Lorentz hyperboloid. Abstract scene-level concepts reside near the apex; representations grow increasingly fine-grained with depth, with geodesics encoding parent–child relationships. This geometric property motivates our analysis of object-centric slot representations through a hyperbolic lens.

Figure 2: Overview of our post-hoc pipeline. Patch features from a frozen DINOv2 backbone are decoded into $N$ slot representations via Slot Attention. The Euclidean path reconstructs slot masks through the baseline decoder, from which ground-truth parent–child pairs $\mathcal{P}$ are derived via mask inclusion. The hyperbolic path projects the same slots onto the Lorentz hyperboloid $\mathbb{H}^{d}_{K}$ via the exponential map, where geodesic distances and Lorentz norms are used for hierarchical analysis without any modification to the original training.

Panel of Figure 3 — Figure 3: Distribution of 300 videos frames ( 3 , 3 ) and 5000 images ( 3 ) of distances from centroid to origin across levels $N\in\{3,5,7,11,13\}$ for three baselines, and level-wise overlap score OV. Lower OV indicates better level separation. $c{=}0.5$ offers the best separation-to-compression tradeoff; higher curvatures achieve tighter but increasingly compressed distributions. The depth ordering is consistently inverted across all models: coarse slots lie furthest from the origin, fine slots closest.

Figure 4: Pairwise agreement between hierarchical orderings induced by Euclidean, Lorentz at $c\in\{0.2,0.5,1.0\}$ , and mask-based ground truth (GT) for SlotContrast. All manifolds agree substantially with GT ( ${\approx}0.75$ ) and Lorentz manifolds are mutually consistent ( $0.89$ – $0.94$ ), suggesting that both geometries capture the broad spatial organisation of the scene.

Model	Manifold	$N=3$	$N=5$	$N=7$	$N=11$	$N=13$
SlotContrast (video)	$\\|\mathbf{s}\\|_{2}$	1.446 $\pm$ 0.127	1.254 $\pm$ 0.137	1.193 $\pm$ 0.138	1.146 $\pm$ 0.140	1.137 $\pm$ 0.142
	$\mathbb{H}^{d}_{c=0.2}$	1.089 $\pm$ 0.109	0.920 $\pm$ 0.105	0.858 $\pm$ 0.101	0.802 $\pm$ 0.100	0.789 $\pm$ 0.100
	$\mathbb{H}^{d}_{c=0.5}$	0.817 $\pm$ 0.089	0.682 $\pm$ 0.079	0.630 $\pm$ 0.074	0.582 $\pm$ 0.073	0.570 $\pm$ 0.073
	$\mathbb{H}^{d}_{c=1}$	0.609 $\pm$ 0.068	0.506 $\pm$ 0.059	0.465 $\pm$ 0.055	0.428 $\pm$ 0.055	0.418 $\pm$ 0.054
VideoSAURv2(video)	$\\|\mathbf{s}\\|_{2}$	5.995 $\pm$ 0.192	5.835 $\pm$ 0.191	5.776 $\pm$ 0.195	5.741 $\pm$ 0.205	5.738 $\pm$ 0.205
	$\mathbb{H}^{d}_{c=0.2}$	3.474 $\pm$ 0.220	3.100 $\pm$ 0.162	2.982 $\pm$ 0.146	2.906 $\pm$ 0.137	2.892 $\pm$ 0.139

Table 1: Slot norms (mean

\,\pm\,

std) in Euclidean (

\ell_{2}

norm of raw slots) and Lorentz hyperbolic spaces (

c\in\{0.2,0.5,1\}

) across granularity levels

N\in\{3,5,7,11,13\}

. Norms decrease monotonically with

N

across all models, consistent with the inverted hierarchy , observed in Figure 3 , which is exactly opposite of expected behaviour as presented in Figure 1 .

Figure 5: Distribution of normalized worst-case Gromov

\delta

hyperbolicity from the slot embeddings of 300 videos of SlotContrast test set. A lower score (closer to 0) indicates that the pairwise distances between embedding encode tree-like structures. The progression shows how the score decreases as the curvature of the manifold increases.

		Hit@1 (%) $\uparrow$
Model	Manifold	$3{\to}5$	$5{\to}7$	$7{\to}11$	$11{\to}13$
SlotContrast	Euclidean	77.2	79.7	72.4	70.5
	$\mathbb{H}^{d}_{c=0.2}$	77.8	79.9	71.8	70.2
	$\mathbb{H}^{d}_{c=0.5}$	77.2	78.8	71.1	69.9
	$\mathbb{H}^{d}_{c=1.0}$	76.1	77.4	69.3	68.7
VideoSAURv2	Euclidean	91.5	93.2	88.6	88.1

Table 2: Parent slot retrieval (Hit@1, %) across consecutive level transitions, comparing Euclidean cosine distance and Lorentz geodesic at

c\in\{0.2,0.5,1.0\}

. Hit@1 measures the percentage of fine slots whose GT parent coarse slot is retrieved at rank 1. Best result per row in bold .

Figure 6: Parent slot retrieval for two-level transitions on a YTVIS video example for SlotContrast. Columns show the input, parent mask, child mask, GT assignment, Euclidean prediction, and Lorentz ( $c{=}0.2$ ) prediction; correct predictions are highlighted in green and incorrect in orange . Although Lorentz predictions are incorrect in some cases by the spatial GT criterion, they are semantically more meaningful, grouping slots by representational abstraction rather than spatial containment. Best viewed in colour.

Hierarchical Representation Learning

Hierarchical representation learning organizes data into nested, coarse-to-fine features that capture parts, patterns, and global structure.

See all 226 papers

Mar 25, 2026

Light Cones For Vision: Simple Causal Priors For Visual Hierarchy

Standard vision models treat objects as independent points in Euclidean space, unable to capture hierarchical structure like parts within wholes. We introduce Worldline Slot Attention, which models objects as persistent trajectories through spacetime worldlines, where each object has multiple slots at different hierarchy levels sharing the same spatial position but differing in temporal coordinates. This architecture consistently fails without geometric structure: Euclidean worldlines achieve 0.078 level accuracy, below random chance (0.33), while Lorentzian worldlines achieve 0.479-0.661 across three datasets: a 6x improvement replicated over 20+ independent runs. Lorentzian geometry also outperforms hyperbolic embeddings showing visual hierarchies require causal structure (temporal dependency) rather than tree structure (radial branching). Our results demonstrate that hierarchical object discovery requires geometric structure encoding asymmetric causality, an inductive bias absent from Euclidean space but natural to Lorentzian light cones, achieved with only 11K parameters. The code is available at: https://github.com/iclrsubmissiongram/loco.

Manglam Kartik, Neel Tushar Shah

Indian Institute of Technology Bombay, Mumbai, India

Hierarchical Representation Learning Object-Centric Representation Learning Slot Attention

Ask PDF

Apr 19, 2026

HSG: Hyperbolic Scene Graph

Scene graph representations enable structured visual understanding by modeling objects and their relationships, and have been widely used for multiview and 3D scene reasoning. Existing methods such as MSG learn scene graph embeddings in Euclidean space using contrastive learning and attention based association. However, Euclidean geometry does not explicitly capture hierarchical entailment relationships between places and objects, limiting the structural consistency of learned representations. To address this, we propose Hyperbolic Scene Graph (HSG), which learns scene graph embeddings in hyperbolic space where hierarchical relationships are naturally encoded through geometric distance. Our results show that HSG improves hierarchical structure quality while maintaining strong retrieval performance. The largest gains are observed in graph level metrics: HSG achieves a PP IoU of 33.17 and the highest Graph IoU of 33.51, outperforming the best AoMSG variant (25.37) by 8.14, highlighting the effectiveness of hyperbolic representation learning for scene graph modeling. Code: https://github.com/AIGeeksGroup/HSG.

Liyang Wang, Zeyu Zhang, Hao Tang

School of Computer Science, Peking University

Graph Representation Learning Hierarchical Representation Learning Graph-Based Retrieval 3D Scene Representation Hyperbolic Representation Learning

Ask PDF

Jul 9, 2026

HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition

Slot attention is a powerful framework for object-centric learning, decomposing visual scenes into latent slots through iterative competitive attention. However, existing methods share two critical limitations: they decompose scenes into a flat set of slots at a single granularity, and this decomposition is based on appearance rather than semantics. Yet humans understand scenes through semantic hierarchies: separating foreground from background, recognizing object categories, and identifying individual instances. Crucially, such semantic hierarchies cannot emerge without supervision, because category names are human constructs, not visual patterns. We propose Hierarchical Slot Attention (HSA), which learns multi-granularity semantic scene decomposition from a single model. HSA decomposes scenes at three levels: holistic (foreground/background), semantic (object categories), and panoptic (individual instances). Using only 10% labeled data, combined with hierarchical alignment loss, HSA learns all three levels jointly. We further introduce grouping purity and containment to measure whether the hierarchy is encoded in representation space, not just output masks. Experiments on COCO and PASCAL VOC demonstrate that HSA outperforms the strongest flat baseline by up to \textbf{

+

41.5} ARI at holistic, \textbf{

+

14.6} at semantic, and \textbf{

+

10.4} at panoptic level on COCO, with even larger gains on Pascal VOC, while requiring a single model instead of three. Code will be made available upon acceptance.

Neelu Madan, Rongzhen Zhao, Andreas Mogelmose +4

1Aalborg University, Denmark · 2Pioneer Centre for AI, Denmark · 3Aalto University, Finland +2

Panoptic Segmentation Hierarchical Representation Learning Object-Centric Representation Learning Slot Attention

Ask PDF

Abstract

Hierarchical Representation Learning

Explore similar work

Light Cones For Vision: Simple Causal Priors For Visual Hierarchy

HSG: Hyperbolic Scene Graph

HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition