A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations

Neelu Madan; Àlex Pujol; Andreas Møgelmose; Sergio Escalera; Kamal Nasrollahi; Graham W. Taylor; Thomas B. Moeslund

cs.CVMar 14, 2026

A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations

Authors: Neelu Madan, Àlex Pujol, Andreas Møgelmose, Sergio Escalera, Kamal Nasrollahi, Graham W. Taylor, Thomas B. Moeslund

Organizations: Aalborg University & Pioneer Centre for AI, Denmark · Universitat de Barcelona & Computer Vision Center, Spain · Milestone Systems, Denmark · University of Guelph & Vector Institute, Canada

Abstract

Slot attention has emerged as a powerful framework for unsupervised object-centric learning, decomposing visual scenes into a small set of compact vector representations called slots, each capturing a distinct region or object. However, these slots are learned in Euclidean space, which provides no geometric inductive bias for the hierarchical relationships that naturally structure visual scenes. In this work, we propose a simple post-hoc pipeline to project Euclidean slot embeddings onto the Lorentz hyperboloid of hyperbolic space, without modifying the underlying training pipeline. We construct five-level visual hierarchies directly from slot attention masks and analyse whether hyperbolic geometry reveals latent hierarchical structure that remains invisible in Euclidean space. Integrating our pipeline with SPOT (images), VideoSAUR (video), and SlotContrast (video), We find that hyperbolic projection exposes a consistent scene-level to object-level organisation, where coarse slots occupy greater manifold depth than fine slots, which is absent in Euclidean space. We further identify a "curvature--task tradeoff": low curvature ( $c{=}0.2$ ) matches or outperforms Euclidean on parent slot retrieval, while moderate curvature ( $c{=}0.5$ ) achieves better inter-level separation. Together, these findings suggest that slot representations already encode latent hierarchy that hyperbolic geometry reveals, motivating end-to-end hyperbolic training as a natural next step. Code and models are available at github.com/NeeluMadan/HHS.

Figures & tables

Figure 1: Visual hierarchy on the Lorentz hyperboloid. Abstract scene-level concepts reside near the apex; representations grow increasingly fine-grained with depth, with geodesics encoding parent–child relationships. This geometric property motivates our analysis of object-centric slot representations through a hyperbolic lens.

Figure 2: Overview of our post-hoc pipeline. Patch features from a frozen DINOv2 backbone are decoded into $N$ slot representations via Slot Attention. The Euclidean path reconstructs slot masks through the baseline decoder, from which ground-truth parent–child pairs $\mathcal{P}$ are derived via mask inclusion. The hyperbolic path projects the same slots onto the Lorentz hyperboloid $\mathbb{H}^{d}_{K}$ via the exponential map, where geodesic distances and Lorentz norms are used for hierarchical analysis without any modification to the original training.

Panel of Figure 3 — Figure 3: Distribution of 300 videos frames ( 3 , 3 ) and 5000 images ( 3 ) of distances from centroid to origin across levels $N\in\{3,5,7,11,13\}$ for three baselines, and level-wise overlap score OV. Lower OV indicates better level separation. $c{=}0.5$ offers the best separation-to-compression tradeoff; higher curvatures achieve tighter but increasingly compressed distributions. The depth ordering is consistently inverted across all models: coarse slots lie furthest from the origin, fine slots closest.

Figure 4: Pairwise agreement between hierarchical orderings induced by Euclidean, Lorentz at $c\in\{0.2,0.5,1.0\}$ , and mask-based ground truth (GT) for SlotContrast. All manifolds agree substantially with GT ( ${\approx}0.75$ ) and Lorentz manifolds are mutually consistent ( $0.89$ – $0.94$ ), suggesting that both geometries capture the broad spatial organisation of the scene.

Model	Manifold	$N=3$	$N=5$	$N=7$	$N=11$	$N=13$
SlotContrast (video)	$\\|\mathbf{s}\\|_{2}$	1.446 $\pm$ 0.127	1.254 $\pm$ 0.137	1.193 $\pm$ 0.138	1.146 $\pm$ 0.140	1.137 $\pm$ 0.142
	$\mathbb{H}^{d}_{c=0.2}$	1.089 $\pm$ 0.109	0.920 $\pm$ 0.105	0.858 $\pm$ 0.101	0.802 $\pm$ 0.100	0.789 $\pm$ 0.100
	$\mathbb{H}^{d}_{c=0.5}$	0.817 $\pm$ 0.089	0.682 $\pm$ 0.079	0.630 $\pm$ 0.074	0.582 $\pm$ 0.073	0.570 $\pm$ 0.073
	$\mathbb{H}^{d}_{c=1}$	0.609 $\pm$ 0.068	0.506 $\pm$ 0.059	0.465 $\pm$ 0.055	0.428 $\pm$ 0.055	0.418 $\pm$ 0.054
VideoSAURv2(video)	$\\|\mathbf{s}\\|_{2}$	5.995 $\pm$ 0.192	5.835 $\pm$ 0.191	5.776 $\pm$ 0.195	5.741 $\pm$ 0.205	5.738 $\pm$ 0.205
	$\mathbb{H}^{d}_{c=0.2}$	3.474 $\pm$ 0.220	3.100 $\pm$ 0.162	2.982 $\pm$ 0.146	2.906 $\pm$ 0.137	2.892 $\pm$ 0.139

Table 1: Slot norms (mean

\,\pm\,

std) in Euclidean (

\ell_{2}

norm of raw slots) and Lorentz hyperbolic spaces (

c\in\{0.2,0.5,1\}

) across granularity levels

N\in\{3,5,7,11,13\}

. Norms decrease monotonically with

N

across all models, consistent with the inverted hierarchy , observed in Figure 3 , which is exactly opposite of expected behaviour as presented in Figure 1 .

Figure 5: Distribution of normalized worst-case Gromov

\delta

hyperbolicity from the slot embeddings of 300 videos of SlotContrast test set. A lower score (closer to 0) indicates that the pairwise distances between embedding encode tree-like structures. The progression shows how the score decreases as the curvature of the manifold increases.

		Hit@1 (%) $\uparrow$
Model	Manifold	$3{\to}5$	$5{\to}7$	$7{\to}11$	$11{\to}13$
SlotContrast	Euclidean	77.2	79.7	72.4	70.5
	$\mathbb{H}^{d}_{c=0.2}$	77.8	79.9	71.8	70.2
	$\mathbb{H}^{d}_{c=0.5}$	77.2	78.8	71.1	69.9
	$\mathbb{H}^{d}_{c=1.0}$	76.1	77.4	69.3	68.7
VideoSAURv2	Euclidean	91.5	93.2	88.6	88.1

Table 2: Parent slot retrieval (Hit@1, %) across consecutive level transitions, comparing Euclidean cosine distance and Lorentz geodesic at

c\in\{0.2,0.5,1.0\}

. Hit@1 measures the percentage of fine slots whose GT parent coarse slot is retrieved at rank 1. Best result per row in bold .

Figure 6: Parent slot retrieval for two-level transitions on a YTVIS video example for SlotContrast. Columns show the input, parent mask, child mask, GT assignment, Euclidean prediction, and Lorentz ( $c{=}0.2$ ) prediction; correct predictions are highlighted in green and incorrect in orange . Although Lorentz predictions are incorrect in some cases by the spatial GT criterion, they are semantically more meaningful, grouping slots by representational abstraction rather than spatial containment. Best viewed in colour.

Hierarchical Representation Learning

Hierarchical representation learning organizes data into nested, coarse-to-fine features that capture parts, patterns, and global structure.

See all 226 papers

Light Cones For Vision: Simple Causal Priors For Visual Hierarchy
Mar 25, 2026Manglam Kartik, Neel Tushar ShahHierarchical Representation Learning Object-Centric Representation Learning
Indian Institute of Technology Bombay, Mumbai, India
HSG: Hyperbolic Scene Graph
Apr 19, 2026Liyang Wang, Zeyu Zhang, Hao TangGraph Representation Learning Hierarchical Representation Learning
School of Computer Science, Peking University
HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition
Jul 9, 2026Neelu Madan, Rongzhen Zhao, Andreas Mogelmose +4Panoptic Segmentation Hierarchical Representation Learning
1Aalborg University, Denmark · 2Pioneer Centre for AI, Denmark · 3Aalto University, Finland +2

Mar 25, 2026J/K move · Enter open · S save

Light Cones For Vision: Simple Causal Priors For Visual Hierarchy

Manglam Kartik, Neel Tushar Shah

Indian Institute of Technology Bombay, Mumbai, India

Hierarchical Representation Learning Object-Centric Representation Learning Slot Attention

Open paper AskPDF arXiv

Standard vision models treat objects as independent points in Euclidean space, unable to capture hierarchical structure like parts within wholes. We introduce Worldline Slot Attention, which models objects as persistent trajectories through spacetime worldlines, where each object has multiple slots at different hierarchy levels sharing the same spatial position but differing in temporal coordinates. This architecture consistently fails without geometric structure: Euclidean worldlines achieve 0.078 level accuracy, below random chance (0.33), while Lorentzian worldlines achieve 0.479-0.661 across three datasets: a 6x improvement replicated over 20+ independent runs. Lorentzian geometry also outperforms hyperbolic embeddings showing visual hierarchies require causal structure (temporal dependency) rather than tree structure (radial branching). Our results demonstrate that hierarchical object discovery requires geometric structure encoding asymmetric causality, an inductive bias absent from Euclidean space but natural to Lorentzian light cones, achieved with only 11K parameters. The code is available at: https://github.com/iclrsubmissiongram/loco.

Abstract

Hierarchical Representation Learning

Explore similar work

Light Cones For Vision: Simple Causal Priors For Visual Hierarchy

HSG: Hyperbolic Scene Graph

HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition