Separating Angular Alignment from Radial Ordering in Hyperbolic Vision-Language Models

Mingyang Gao; Yuwei Wu; Shiyu Luo; Wei Wu; Xiaomeng Fan; Qihao Lv; Haoyu Zhang; Rui Gao

cs.CVOct 23, 2026

Separating Angular Alignment from Radial Ordering in Hyperbolic Vision-Language Models

Authors: Mingyang Gao, Yuwei Wu, Shiyu Luo, Wei Wu, Xiaomeng Fan, Qihao Lv, Haoyu Zhang, Rui Gao,

Organizations: Beijing Institute of Technology · Shenzhen MSU-BIT University

Abstract

Hyperbolic vision-language models are well suited to modeling the hierarchical structure of multimodal data, because hyperbolic volume grows exponentially with radius, enabling low-distortion representations of tree-like taxonomies from general to specific concepts. However, existing pretraining objectives for hyperbolic vision-language models entangle semantic alignment with entailment modeling, forcing radial coordinates to simultaneously encode semantic matching and hierarchical structure, leading to conflicting geometric constraints. In this paper, we introduce AURORA, a hyperbolic vision-language framework that decouples the objectives through angular alignment and a novel Radial-Angular Order Cone (RAOC) for entailment modeling. Angular alignment captures semantic similarity while preserving the radial structure for representing hierarchies. Unlike conventional entailment cones, RAOC determines angular allowance from relative parent-child radial gaps. It is incorporated into hierarchical entailment contrastive training to capture partial-order relations. We further introduce a norm regularization term that prevents undesirable radial degeneration for stable training. We train models on millions of image-text pairs and evaluate them across zero-shot classification, image-text retrieval, hierarchical reasoning, and compositional understanding. Our model consistently outperforms strong Euclidean and hyperbolic vision-language baselines, demonstrating the effectiveness of decoupling semantic alignment from hierarchical entailment modeling.

Figures & tables

Figure 1: Motivation. Top: Prior hyperbolic VLMs pull paired embeddings together, disrupting radial ordering and entailment. The apple image can even lie closer to the Lorentz origin than “apple”. Bottom: AURORA preserves radial separation and cleaner entailment: the rocket returns to its own cone, incorrect relations are reduced, and a coherent hierarchy emerges.

Figure 2: Overview of Aurora . Root-angle alignment brings related visual and textual semantics together, while HEC training via RAOC organizes concepts radially from general to specific.

Figure 3: RAOC. Greater angular separation requires a larger radial gap.

ViT-S/16		General datasets						Fine-grained datasets						Specialized datasets
Model	w/ boxes	ImageNet	CIFAR-10	CIFAR-100	SUN397	Caltech-101	STL-10	Food-101	CUB	Cars	Aircraft	Pets	Flowers	DTD	EuroSAT	RESISC45	Country211
CLIP	✗	32.66	74.19	42.22	42.91	69.80	90.09	42.72	12.55	3.93	2.06	43.22	18.03	17.57	25.47	33.64	3.88
CLIP	✓	32.49	73.69	42.15	42.66	70.07	89.83	42.50	12.48	4.01	2.10	43.28	17.79	17.61	26.04	33.37	3.84
SigLIP	✗	34.30	66.57	38.38	44.73	71.42	91.75	38.98	13.33	4.89	2.13	46.65	23.17	18.40	32.63	35.42	4.15
SigLIP	✓	34.34	67.07	38.50	44.82	71.55	91.82	39.27	13.33	4.92	2.24	46.56	23.03	18.36	31.42	35.37	4.23
MERU	✗	31.74	76.08	41.24	41.28	67.76	87.24	41.09	11.97	3.80	2.88	42.53	16.75	17.46	27.75	33.88	3.82

Table 1: Zero-shot image classification on ViT-S/16. Results are reported as top-1 accuracy (%). Bold indicates the best results. Underlining denotes the second best.

ViT-S/16		Text $\rightarrow$ Image				Image $\rightarrow$ Text				Hierarchical Classification
Model		COCO		Flickr		COCO		Flickr		WordNet
	w/ boxes	R@5	R@10	R@5	R@10	R@5	R@10	R@5	R@10	TIE $\downarrow$	LCA $\downarrow$	$J\uparrow$	$P_{H}\uparrow$	$R_{H}\uparrow$
CLIP	✗	50.31	61.52	76.98	84.48	65.75	76.15	87.10	93.00	4.43	2.51	0.72	0.79	0.80
CLIP	✓	50.27	61.47	76.75	84.42	65.49	76.08	87.20	93.17	4.42	2.50	0.72	0.78	0.80
SigLIP	✗	52.32	63.90	77.54	85.24	65.98	76.78	88.60	93.23	4.22	2.47	0.75	0.83	0.82
SigLIP	✓	52.27	63.81	77.21	84.09	66.03	76.84	88.60	93.40	4.23	2.47	0.74	0.83	0.83

Table 2: Zero-shot retrieval and hierarchical classification on ViT-S/16. Retrieval performance is reported as Recall@5/10 (%). Hierarchical metrics are shown to two decimal places, with rankings determined before rounding. Bold indicates the best results. Underlining denotes the second best.

ViT-S/16		VL-CheckList-Object						SugarCrepe
Model		Location			Size			Replace			Swap		Add
	w/ boxes	Center	Mid	Margin	Large	Medium	Small	Obj	Att	Rel	Obj	Att	Obj	Att	Overall
CLIP	✗	85.10	79.35	71.22	84.48	74.61	65.72	86.74	78.00	69.04	59.49	65.27	77.53	70.76	75.71
CLIP	✓	85.24	78.41	70.55	85.32	75.02	64.67	85.94	77.45	69.94	59.89	64.41	77.90	70.76	75.69
SigLIP	✗	89.33	83.04	74.91	88.30	78.50	68.82	87.35	78.55	69.49	60.82	65.77	78.08	71.10	76.26
SigLIP	✓	89.83	83.03	74.50	88.13	78.13	69.47	87.01	78.43	69.18	61.38	66.67	78.03	70.28	76.12

Table 3: Hard-negative compositional classification on ViT-S/16. Models are evaluated on VL-CheckList-Object and SugarCrepe. Bold indicates the best results. Underlining denotes the second best.

Panel of Figure 4 — Figure 4: Representation geometry with and without RAOC. (a) Spatial-norm distributions and HoroPCA projections (b) without and (c) with RAOC for AURORA ViT-S. RAOC yields more distinct norm distributions and broader projected organization, with boxes tending to lie nearer the projected origin than full inputs.

Panel of Figure 5 — Figure 5: Local hierarchical organization with and without RAOC. AURORA ViT-S (a) w/o cone and (b) w/ cone for the text query “bag.” Scatter plots show radial gaps and root angles for visual regions, semantic siblings, and unrelated controls; the corresponding visual hierarchy is shown on the right. RAOC produces clearer query-relative organization along the relevant hierarchy branch.

AURORA-ViT-S/16					Classification		COCO R@5		Hierarchy
ID	Alignment score	Order objective	Norm	Loss	ImageNet	Food-101	T $\rightarrow$ I	I $\rightarrow$ T	TIE $\downarrow$	$J\uparrow$
(a)	root angle	EC	✓	Sigmoid	38.12	45.92	53.48	66.72	3.79	0.76
(b)	root angle	RAOC	✗	Sigmoid	40.21	48.57	54.98	68.41	3.70	0.77
(c)	root angle	RAOC	✓	InfoNCE	39.42	47.71	54.43	67.93	3.76	0.76
(d)	root angle	none	✓	Sigmoid	40.27	47.06	55.29	68.92	3.74	0.77
(e)	root angle	none	✗	Sigmoid	38.91	45.38	54.56	67.88	3.85	0.76

Table 4: Ablation study of AURORA-ViT-S/16 pre-trained on grounded GRIT, evaluated on image classification, COCO retrieval, and hierarchical classification. Controlled variants isolate the contribution of individual training components and loss terms.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning	Role
$c$	positive curvature magnitude; manifold curvature is $-c$	learned global scalar
$\tilde{\mathbf{x}}\in\mathbb{R}^{d}$	spatial component of a Lorentz point $\mathbf{x}$	Lorentz coordinate
$r_{x}=\\|\tilde{\mathbf{x}}\\|$	Lorentz spatial norm	radial regularization variable
$\rho_{x}=\sqrt{c}\,d_{c}(O,\mathbf{x})$	scaled distance from the origin	radial order coordinate
$\mathbf{u}_{x}=\tilde{\mathbf{x}}/\\|\tilde{\mathbf{x}}\\|$	spatial direction of non-origin $\mathbf{x}$	angular coordinate
$\mathbf{w}_{x}=\Phi_{c}(\mathbf{x})$	Poincaré coordinate of $\mathbf{x}$	equivalent coordinate view

Appendix

Table 5: Core notation used in the geometric analysis.

Model	GPUs	Peak VRAM/GPU (GB)	Train time (h)
HyCoCLIP	$4\times$ A40	39.4	91.80
PHyCLIP	$4\times$ A40	40.9	93.40
AURORA-InfoNCE	$4\times$ A40	21.2	80.83
AURORA	$4\times$ A40	13.8	77.51

Appendix

Table 6: ViT-S/16 training efficiency in the common four-A40 environment. Values are means over three runs with randomly selected seeds.

			General datasets						Fine-grained datasets						Specialized datasets
Backbone	Model	w/ boxes	ImageNet	CIFAR-10	CIFAR-100	SUN397	Caltech-101	STL-10	Food-101	CUB	Cars	Aircraft	Pets	Flowers	DTD	EuroSAT	RESISC45	Country211
ViT S/16	CLIP	✗	32.66	74.19	42.22	42.91	69.80	90.09	42.72	12.55	3.93	2.06	43.22	18.03	17.57	25.47	33.64	3.88
	CLIP	✓	32.49	73.69	42.15	42.66	70.07	89.83	42.50	12.48	4.01	2.10	43.28	17.79	17.61	26.04	33.37	3.84
	SigLIP	✗	34.30	66.57	38.38	44.73	71.42	91.75	38.98	13.33	4.89	2.13	46.65	23.17	18.40	32.63	35.42	4.15
	SigLIP	✓	34.34	67.07	38.50	44.82	71.55	91.82	39.27	13.33	4.92	2.24	46.56	23.03	18.36	31.42	35.37	4.23
	MERU	✗	31.74	76.08	41.24	41.28	67.76	87.24	41.09	11.97	3.80	2.88	42.53	16.75	17.46	27.75	33.88	3.82

Appendix

Table 7: Complete zero-shot image classification comparison for ViT-S/16 and ViT-B/16. Top-1 accuracy (%). ViT-S/16 and ViT-B/16 best results are shown in bold and second-best results are underlined .

			Text $\rightarrow$ Image				Image $\rightarrow$ Text				Hierarchical Classification
Backbone	Model	w/ boxes	COCO		Flickr		COCO		Flickr		WordNet
			R@5	R@10	R@5	R@10	R@5	R@10	R@5	R@10	TIE $\downarrow$	LCA $\downarrow$	$J\uparrow$	$P_{H}\uparrow$	$R_{H}\uparrow$
ViT S/16	CLIP	✗	50.31	61.52	76.98	84.48	65.75	76.15	87.10	93.00	4.43	2.51	0.72	0.79	0.80
	CLIP	✓	50.27	61.47	76.75	84.42	65.49	76.08	87.20	93.17	4.42	2.50	0.72	0.78	0.80
	SigLIP	✗	52.32	63.90	77.54	85.24	65.98	76.78	88.60	93.23	4.22	2.47	0.75	0.83	0.82
	SigLIP	✓	52.27	63.81	77.21	84.09	66.03	76.84	88.60	93.40	4.23	2.47	0.74	0.83	0.83

Appendix

Table 8: Complete zero-shot retrieval and hierarchical classification comparison for ViT-S/16 and ViT-B/16. Retrieval results are Recall@5/10 (%). Hierarchical metrics are rounded to two decimal places for display; best and second-best markings follow the underlying values.

			VL-CheckList-Object						SugarCrepe
Backbone	Model	w/ boxes	Location			Size			Replace			Swap		Add
			Center	Mid	Margin	Large	Medium	Small	Obj	Att	Rel	Obj	Att	Obj	Att	Overall
ViT S/16	CLIP	✗	85.10	79.35	71.22	84.48	74.61	65.72	86.74	78.00	69.04	59.49	65.27	77.53	70.76	75.71
	CLIP	✓	85.24	78.41	70.55	85.32	75.02	64.67	85.94	77.45	69.94	59.89	64.41	77.90	70.76	75.69
	SigLIP	✗	89.33	83.04	74.91	88.30	78.50	68.82	87.35	78.55	69.49	60.82	65.77	78.08	71.10	76.26
	SigLIP	✓	89.83	83.03	74.50	88.13	78.13	69.47	87.01	78.43	69.18	61.38	66.67	78.03	70.28	76.12

Appendix

Table 9: Complete hard-negative compositional evaluation for ViT-S/16 and ViT-B/16.

Figure 6: Image to text hierarchy comparisons for five object queries. Rows show electric guitar, flamingo, grand piano, ladybug, and laptop. The left column uses RAOC, and the right column omits it. Dashed blue lines trace the query’s semantic path, while gray lines show related branches. RAOC preserves the angular preference for the query class and gives a consistent radial progression along each path. The radial reversals for electric guitar and ladybug without RAOC illustrate why angular alignment alone does not ensure hierarchical order.

Figure 7: Image to text hierarchy comparisons for five further queries. Rows show leopard, lion, monarch, zebra, and desk, with RAOC on the left and without RAOC on the right. RAOC places the displayed semantic paths in a consistent radial order. Without RAOC, the animal examples exhibit reversals between successive concepts even when the query class remains well aligned. The desk path is ordered in both variants, showing that the effect concerns consistency across examples rather than the absence of all hierarchy without RAOC.

Figure 8: Text to image hierarchy for the query “shoes.” AURORA ViT-S with RAOC is shown above the variant without RAOC. With RAOC, the full image, shoes and tennis ball region, and shoes crop follow a consistent radial progression as their root angles decrease. Without RAOC, the shoes crop remains the most aligned region, but the final step reverses the radial direction. The region tree provides the common reference hierarchy.

Figure 9: Text to image hierarchy for the query “eggplant.” The top panel uses RAOC, and the bottom panel omits it. Both models assign the eggplant crop a smaller root angle than the full image. RAOC also places the full image, eggplant and pumpkin region, and eggplant crop in the expected radial order. Without RAOC, this branch progresses in the opposite radial direction despite retaining its angular preference for eggplant.

Figure 10: Text to image hierarchy for the query “earphone.” The top and bottom panels show the models with and without RAOC, respectively. With RAOC, the full image, earphone and cup region, and earphone crop combine progressively smaller root angles with increasing radial gaps. Without RAOC, the same angular trend is present, but the radial sequence is reversed. The comparison separates semantic alignment from the hierarchical ordering encouraged by RAOC.

Figure 11: Text to image hierarchy for the query “chair.” RAOC is used in the top panel and omitted in the bottom panel. Both variants retain radial progression along the full image, vase and chair region, and chair crop. With RAOC, their root angles also decrease consistently. Without RAOC, the intermediate region has a larger root angle than the full image, revealing less consistent angular organization along this branch.

Figure 12: Text to image hierarchy for the query “bread.” The model with RAOC appears above the model without RAOC. With RAOC, the full image, bread and grape region, and bread crop form a consistent sequence in both radial gap and root angle. Without RAOC, the bread and grape region has a slightly smaller root angle than the bread crop, and the final transition reverses the radial direction. RAOC yields clearer organization along the branch containing the queried object.

Parameter	Classification		COCO R@5		Hierarchy
$\beta$	ImageNet $\uparrow$	Food-101 $\uparrow$	T $\rightarrow$ I $\uparrow$	I $\rightarrow$ T $\uparrow$	TIE $\downarrow$	$J\uparrow$
2	41.31	49.61	55.02	68.71	3.69	0.77
4	41.18	49.82	55.31	68.97	3.67	0.77
8	41.54	50.13	55.77	69.42	3.62	0.78
16	41.12	49.74	55.18	68.84	3.71	0.77
32	41.36	49.96	55.41	69.06	3.66	0.77

Appendix

Table 10: Sensitivity to the RAOC parameter

\beta

. All other hyperparameters remain fixed at their default values.

Parameter	Classification		COCO R@5		Hierarchy
$\lambda_{e}$	ImageNet $\uparrow$	Food-101 $\uparrow$	T $\rightarrow$ I $\uparrow$	I $\rightarrow$ T $\uparrow$	TIE $\downarrow$	$J\uparrow$
0.0375	40.82	48.93	54.72	68.31	3.81	0.76
0.075	41.37	49.88	55.39	69.03	3.68	0.77
0.15	41.54	50.13	55.77	69.42	3.62	0.78
0.30	40.21	48.76	54.65	68.20	3.86	0.76
0.60	39.12	46.94	53.47	66.91	4.05	0.74

Appendix

Table 11: Sensitivity to the entailment weight

\lambda_{e}

. All other hyperparameters remain fixed at their default values.

Parameter	Classification		COCO R@5		Hierarchy
$\lambda_{n}$	ImageNet $\uparrow$	Food-101 $\uparrow$	T $\rightarrow$ I $\uparrow$	I $\rightarrow$ T $\uparrow$	TIE $\downarrow$	$J\uparrow$
0.0125	40.71	49.12	54.89	68.54	3.76	0.76
0.025	41.08	49.77	55.34	69.01	3.69	0.77
0.05	41.54	50.13	55.77	69.42	3.62	0.78
0.10	41.29	49.96	55.46	69.18	3.70	0.77
0.20	40.62	48.85	54.73	68.37	3.88	0.76

Appendix

Table 12: Sensitivity to the norm regularization weight

\lambda_{n}

. All other hyperparameters remain fixed at their default values.

Parameter	Classification		COCO R@5		Hierarchy
$\sigma_{r}$	ImageNet $\uparrow$	Food-101 $\uparrow$	T $\rightarrow$ I $\uparrow$	I $\rightarrow$ T $\uparrow$	TIE $\downarrow$	$J\uparrow$
0.025	41.08	49.74	55.21	68.92	3.70	0.77
0.05	41.32	50.01	55.54	69.18	3.65	0.77
0.10	41.54	50.13	55.77	69.42	3.62	0.78
0.20	41.46	50.04	55.68	69.31	3.64	0.78
0.40	41.38	49.91	55.59	69.20	3.67	0.77

Appendix

Table 13: Sensitivity to the radial margin scale

\sigma_{r}

. All other hyperparameters remain fixed at their default values.

Vision-Language Models

Vision-language models connect visual inputs with text for retrieval, question answering, generation, and grounded multimodal reasoning.

See all 738 papers

ARGENT: Adaptive Hierarchical Image-Text Representations
Mar 24, 2026Chuong Huynh, Hossein Souri, Abhinav Kumar +3Vision-Language Models Hierarchical Representation Learning
University of Maryland, College Park · Samsung Research America, AI Center – Mountain View
Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models
Jul 6, 2026Jaeyoung Kim, Eunseok Kim, Dongsuk JangVLM Evaluation Vision-Language Models
MADI
Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models
Mar 23, 2026Hayeon Kim, Ji Ha Jang, Junghun James Kim +1Vision-Language Models Hierarchical Representation Learning
Dept. of Electrical and Computer Engineering, Seoul National University, Republic of Korea · INMC & IPAI, Seoul National University, Republic of Korea

Mar 24, 2026J/K move · Enter open · S save

ARGENT: Adaptive Hierarchical Image-Text Representations

Chuong Huynh, Hossein Souri, Abhinav Kumar, Vitali Petsiuk+2

University of Maryland, College Park · Samsung Research America, AI Center – Mountain View

Vision-Language Models Hierarchical Representation Learning Hyperbolic Representation Learning VLM Evaluation

Open paper AskPDF arXiv

Large-scale Vision-Language Models (VLMs) such as CLIP learn powerful semantic representations but operate in Euclidean space, which fails to capture the inherent hierarchical structure of visual and linguistic concepts. Hyperbolic geometry, with its exponential volume growth, offers a principled alternative for embedding such hierarchies with low distortion. However, existing hyperbolic VLMs use entailment losses that are unstable: as parent embeddings contract toward the origin, their entailment cones widen toward a half-space, causing catastrophic cone collapse that destroys the intended hierarchy. Additionally, hierarchical evaluation of these models remains unreliable, being largely retrieval-based and correlation-based metrics and prone to taxonomy dependence and ambiguous negatives. To address these limitations, we propose an adaptive entailment loss paired with a norm regularizer that prevents cone collapse without heuristic aperture clipping. We further introduce an angle-based probabilistic entailment protocol (PEP) for evaluating hierarchical understanding, scored with AUC-ROC and Average Precision. This paper introduces a stronger hyperbolic VLM baseline ARGENT, Adaptive hieRarchical imaGe-tExt represeNTation. ARGENT improves the SOTA hyperbolic VLM by 0.7, 1.1, and 0.8 absolute points on image classification, text-to-image retrieval, and proposed hierarchical metrics, respectively.

Abstract

Vision-Language Models

Explore similar work

ARGENT: Adaptive Hierarchical Image-Text Representations

Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models

Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models