We study a tree-structured regularizer over class-prototype layouts in a hierarchical-classification model and ask whether the choice of latent manifold for the prototypes (Euclidean R^d vs. the Poincare ball B^d_c) affects how well that regularizer can be satisfied without distorting the data likelihood. The two manifolds differ only in their volume growth: hyperbolic space grows exponentially with radius and embeds trees with provably lower distortion than R^d of matched dimension, so the structured regularizer should be cheaper to satisfy on B^d_c. Across 150 seed-replicated regularized maximum-likelihood fits spanning embedding dimension, curvature, and regularizer strength on WikiArt (27 styles, 81,446 paintings, frozen CLIP ViT-B/16 features), we find a single robust effect: Poincare prototypes preserve the topology of the nearest-neighbor graph in latent space substantially better than matched Euclidean prototypes (sibling recall@5 +8.7 pp, cousin recall +15.2 pp; paired-t p < 10^-4, sign agreement 0.94), and the gap holds across three reference-tree definitions (hand-built lineage, CLIP-derived, and DINOv2-derived). On classification, Euclidean prototypes are tied with logistic regression on raw encoder features, indicating no detectable contribution from the latent geometry; only the hyperbolic fit improves on a k-NN encoder baseline for local retrieval. Global tree-fidelity comparisons are unstable across reference trees and we do not claim a winner. The results give an empirical separation, on a real hierarchical-classification problem, between two natural latent geometries for a class-structured regularizer.
Figures & tables
Figure 1: Prototype layouts at d=2 , best curvature per geometry. Each large dot is a learned style prototype; small dots are a 600-image sample of validation embeddings. The dashed circle on the right marks the Poincaré ball boundary at radius 1/c . Hyperbolic prototypes settle into a near-boundary ring (mean radius 0.561 , std 0.013 across all 27 styles, against a boundary at 0.577 ). This is the exponential-volume regime where the manifold has the capacity for trees ( Nickel and Kiela, 2017 ; Sala et al., 2018 ) . Euclidean prototypes scatter diffusely with no analogous structural pressure (mean radius 2.96 , std 0.995 ). The geometry behaves as theory predicts; the rest of the paper measures whether that geometric difference shows up in downstream metrics.
euclidean
hyperbolic
logistic-on-CLIP
kNN-5-on-CLIP
metric
Top-1
0.659 ± 0.003
0.606 ± 0.002
0.641
0.635
Top-5
0.959 ± 0.001
0.935 ± 0.001
—
—
Balanced acc.
0.554 ± 0.002
0.463 ± 0.004
0.611
0.588
Class-center / tree Spearman
0.350 ± 0.027
0.242 ± 0.002
—
—
Avg. tree distortion
1.742 ± 0.009
1.861 ± 0.003
—
—
Table 1: Best Euclidean vs. best hyperbolic configuration across the dimension and regularizer-strength sweeps, selected per geometry by mean top-1 across seeds. Mean ± seed standard deviation. Two rightmost columns: logistic regression and k -NN-5 on raw CLIP features (no learned head, no prototype objective). Euclidean leads classification and global-tree metrics over hyperbolic prototypes; hyperbolic prototypes lead sibling and cousin recall by 4 – 5 pp. Read against the calibration baselines, the Euclidean fit is tied with logistic on top-1 and with k -NN on sibling recall; only the hyperbolic fit improves on k -NN for sibling and cousin recall.
Figure 2: Metrics versus embedding dimension d (90 runs; three seeds per config; hyperbolic at the best curvature per d per seed). Top-1 saturates near d=16 for both geometries with Euclidean ahead by 4 – 6 pp. Sibling recall@5 decreases with d for Euclidean (from 0.221 at d=2 to 0.142 at d=64 ) but stays nearly flat for hyperbolic, opening a 5 pp gap by d=64 .
Figure 3: Sibling and cousin recall@5 with relations defined by the default tree (dashed) and a CLIP-derived empirical tree (solid), per geometry per embedding dimension. The hyperbolic lead is preserved under the empirical-tree redefinition; on sibling recall the gap widens slightly. The DINOv2-derived empirical tree is in Appendix F .
Figure 4: Top-1 accuracy versus sibling recall@5 for every sweep configuration. Marker size scales with the regularizer strength λ . Vertical dotted line: logistic-regression top-1 on raw CLIP features ( 64.1% ). Black star: k -NN-5 on raw CLIP features (sibling recall 0.160 ). The Euclidean cluster sits at or below the encoder baseline on retrieval; only the hyperbolic cluster populates the upper region of the plot.
Figure 5: Top-1, prototype-tree Spearman, and sibling recall@5 versus regularizer strength λ at d=8 , c=1 (three seeds per point). The regularizer drives tree-Spearman from ∼0.3 to ∼0.7 in both geometries, but classification collapses for λ≥1 . The hyperbolic top-1 gap never closes; sibling recall is stable for hyperbolic across all λ , while for Euclidean it rises only at the largest tested λ , where classification has already collapsed. Mean tree distortion (omitted here for legibility) decreases uniformly with λ in both geometries; full four-panel version in Appendix D .
Figure 6: Each geometry trained with λ=1 at d=8 against three different reference trees, then evaluated against the default tree. Bars are seed means; whiskers are seed standard deviation. Tree-Spearman against the default tree collapses for non-default training trees; the regularizer only helps the metric anchored to the training tree. The geometry winner does not change on any panel: Euclidean wins top-1 on every tree, hyperbolic wins sibling recall on every tree. Full four-panel version including mean tree distortion in Appendix E .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Per-style image counts on WikiArt-Refined. Impressionism dominates with 13,060 examples; Analytical Cubism, Action Painting, and Synthetic Cubism each have fewer than 250 . The imbalance ratio between extrema is 133.3× .
Figure 8: Tree distance matrices for the three reference hierarchies used in this paper: default lineage (left), chronological era grouping (centre), flat null (right). Rows and columns are styles in the same order across panels.
Figure 9: Full four-panel version of the regularizer-strength sweep (body Figure 5 drops the third panel for legibility). Mean tree distortion decreases monotonically with λ in both geometries, mirroring the rise in tree-Spearman.
Figure 10: Full four-panel version of the training-tree ablation (body Figure 6 drops the third panel for legibility). Tree distortion against the default tree is uniformly higher when training against a non-default tree.
Figure 11: Prototype-distance / tree-distance Spearman against the hand-built default tree (blue) and a CLIP-empirical tree (red). Both geometries’ prototypes align substantially better with the empirical tree than with the hand-built one. Caveat: the empirical tree is built in Euclidean space from the same CLIP features used to train the prototypes, so the alignment with prototype distances is partly expected by construction. The geometry comparison is the load-bearing claim, not the absolute level.
Figure 12: Substituting a DINOv2 ViT-B/14 encoder for CLIP in the empirical-tree construction. Left: prototype-tree Spearman against three references. The hyperbolic global-Spearman lead seen against the CLIP-empirical tree does not replicate against the DINOv2-empirical tree: Euclidean prototypes lead at every d≥4 . Centre: sibling recall@5 against the three reference-tree sibling-set definitions. Hyperbolic leads on every reference at every d≥4 . Right: pairwise-distance Spearman between reference trees. Hand-built correlates only weakly with either encoder-derived tree ( 0.08 , 0.12 ); the two encoder-derived trees correlate 0.50 with each other.
Figure 13: Row-normalized confusion matrices at d=8 , with rows and columns permuted by a depth-first traversal of the default style tree so that hierarchically adjacent styles sit next to each other on both axes. Both models concentrate mistakes in near-diagonal blocks: errors predominantly fall on tree-adjacent styles. The hyperbolic block structure is visibly softer near the diagonal: more confusion mass spreads into immediate siblings, less mass jumps to distant styles. This is the same fact the sibling-recall numbers report, in a form a viewer can absorb at a glance.
Hierarchical graphs embed in hyperbolic space with lower distortion than in Euclidean space owing to its negative curvature. However, their gradient-based learning is hampered at large radii, where the Poincaré ball and the Lorentz hyperboloid models fail numerically. Polar coordinates avoid this problem, but the hyperbolic metric scales the angular step by the hyperbolic sine of the radius, freezing angular motion. We observe that this factor is a choice, silently fixed by existing implementations: the Euclidean tangent parametrization, for instance, uses the radius itself. We show that other choices are not only possible but preferable. They are endpoints of a one-parameter family of optimization preconditioners with curvatures from −1 to 0, while the embedding remains at curvature −1. We show that since the Euclidean preconditioner rearranges a layout but refines it poorly, while an intermediate one refines far better once a layout is in place, combining them in two stages reduces the loss on real-world trees by 46-74% over the best single curvature.
We study the implicit bias of Riemannian gradient flow for hyperbolic multiclass classification with fixed class prototypes in hyperbolic space Hn. Our framework accommodates general permutation invariant relative margin (PERM) losses, a class that includes cross entropy and other standard multiclass losses. Our analysis is based on a decomposition: at large radius, the distance to each prototype splits into a radial term and a direction-dependent term described by the Busemann function. This yields two main results. First, we prove a radial dichotomy: the sign of a drift coefficient μ determines whether the radius is pushed toward the ideal boundary or back toward the interior; if the positive drift persists, then r(t)=21logt+O(1), while persistent negative drift returns the trajectory to the large-radius threshold in finite time. Second, we show that the boundary direction converges to a critical point of the Busemann risk on ∂Hn. These results provide a rigorous asymptotic perspective on two phenomena we refer to as boundary saturation and near-boundary clustering in hyperbolic representation learning.
Hyperbolic spaces provide a natural geometry for representing hierarchical and tree-structured data due to their exponential volume growth. To leverage these benefits, neural networks require intrinsic and efficient components that operate directly in hyperbolic space. In this work, we lift two core components of neural networks, Multinomial Logistic Regression (MLR) and Fully Connected (FC) layers, into hyperbolic space via Busemann functions, resulting in Busemann MLR (BMLR) and Busemann FC (BFC) layers with a unified mathematical interpretation. BMLR provides compact parameters, a point-to-horosphere distance interpretation, batch-efficient computation, and a Euclidean limit, while BFC generalizes FC and activation layers with comparable complexity. Experiments on image classification, genome sequence learning, node classification, and link prediction demonstrate improvements in effectiveness and efficiency over prior hyperbolic layers. The code is available at https://github.com/GitZH-Chen/HBNN.