After the Euclidean Highway: Hyperbolic Expert AI as the Next Innovation

Whether a hyperbolic representation model uses its geometry cannot be inferred from curvature alone: what matters is the dimensionless operating point $\sqrt{c}ρ$ and whether the radial and cone mechanisms are operational there. We develop necessary-condition diagnostics and audit three published hyperbolic vision-language families -- MERU, HyCoCLIP, and PHyCLIP -- across released checkpoints and matched interventions. All converged checkpoints remain near-Euclidean ($H(u)\approx1$; none reaches $\sqrt{c}ρ>1$), and releasing the curvature floor changes $c$ and norms without leaving this regime or substantially degrading downstream performance. Entailment cones are inactive or saturated, and graded traversal fails under controlled readouts, including the models' native distance metrics. External parent-child ordering shows no shuffle-controlled pair-specific radial signal at quantified sensitivity; the only surviving pair-specific signal, a statistically detectable but small residual on the GRIT box-to-full-caption relation, remains non-operative under the evaluated readouts. Taxonomy correlations show no detectable norm contribution beyond cosine, and coarse-retrieval gains co-vary with box/compositional supervision without establishing an active radial mechanism. Gradient diagnostics expose a low-curvature, wide-cone shortcut in the entailment objective. A closed-form aperture identity places the saturation edge at $\sqrt{c}ρ\le2K$: with the floor released, all trained relation-level parent means lie at or below this edge, leaving the parent cones fully or nearly saturated. Entailment-off runs pass the edge and continue contracting. The shortcut is the dominant accelerator of collapse, not its sole cause. These audited formulations do not show an operative radial/cone mechanism under our diagnostics. We distill the audit into a five-number geometry report for hierarchy claims.

Multi-step reasoning remains a central challenge for large language models: single-pass generation is efficient but lacks accuracy; tree-search methods explore multiple paths but are computation-heavy. We address this gap by distilling reasoning progress into a hyperbolic geometric signal that guides step-by-step generation. Our approach is motivated by a structural observation: in combinatorial reasoning trees, solution-bearing states are few while dead ends are exponentially numerous. The hyperbolic space matches this asymmetry, with compact volume near the origin and exponentially expanding capacity toward the boundary, so that distance-to-origin naturally encodes solution proximity while angular separation distinguishes branches requiring different next operations. We train a lightweight head to project LLM hidden states into this space, then fine-tune a low-rank adapter interactively on its own reasoning attempts to act on the injected signal. Across multiple benchmarks, the geometric signal yields consistent gains, with larger improvements on deeper reasoning chains. Our code is publicly available at https://github.com/yuyuliu11037/HyperGuide.

Extending a fully post-trained language model with new domain capabilities is fundamentally limited by monolithic training paradigms: retraining from scratch is expensive and scales poorly, while continued training often degrades existing capabilities. We present BAR (Branch-Adapt-Route), which trains independent domain experts, each through its own mid-training, supervised finetuning, and reinforcement learning pipeline, and composes them via a Mixture-of-Experts architecture with lightweight router training. Unlike retraining approaches that mix all domains and require full reprocessing for any update (with cost scaling quadratically), BAR enables updating individual experts independently with linear cost scaling and no degradation to existing domains. At the 7B scale, with experts for math, code, tool use, and safety, BAR achieves an overall score of 49.1 (averaged across 7 evaluation categories), matching or exceeding re-training baselines (47.8 without mid-training, 50.5 with). We further show that modular training provides a structural advantage: by isolating each domain, it avoids the catastrophic forgetting that occurs when late-stage RL degrades capabilities from earlier training stages, while significantly reducing the cost and complexity of updating or adding a domain. Together, these results suggest that decoupled, expert-based training is a scalable alternative to monolithic retraining for extending language models.