cs.AISep 28, 2026

Building Transformation Layers for Riemannian Neural Networks

Authors: Ziheng Chen

Organizations: University of Trento · MPI for Intelligent Systems, Tübingen

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recently, deep neural networks on manifold-valued representations have garnered significant attention across various machine learning applications. One recent focus is the generalization of Euclidean fully connected (FC) and convolutional layers to non-Euclidean geometries. However, previous approaches typically focus on a few selected manifolds and rely on specific properties of the target manifold. In contrast, this work proposes a framework for constructing FC and convolutional layers over computationally tractable Riemannian spaces. This framework incorporates several previous FC layers across different geometries as special cases and is instantiated on ten representative manifolds, including three hyperbolic models, five geometries of the symmetric positive definite (SPD) manifold, and two Grassmannian perspectives. Experiments on different manifolds demonstrate the effectiveness and applicability of our approach. Code can be found at https://github.com/GitZH-Chen/RieTrans.

Figures & tables

Figure 1 : Riemannian convolution within a receptive field. Here, Fk(⋅)\mathcal{F}^{k}(\cdot) denotes the kk -th FC transformation.
MethodSpaceMechanismReferences
MöbiusPKn\mathbb{P}^{n}_{K}Tangent[ 28 , Def. 3.2]
EinsteinKKn\mathbb{K}^{n}_{K}Tangent[ 51 , Thm. 9]
LFCHKn\mathbb{H}^{n}_{K}Spacetime[ 14 , Eq. (1)]
NestFCHKn\mathbb{H}^{n}_{K}Nested projection[ 26 , Eq. (14)]
Poincaré FCPKn\mathbb{P}^{n}_{K}Poincaré[ 66 , Sec. 3.2]
OursPKn,KKn,HKn\mathbb{P}^{n}_{K},\mathbb{K}^{n}_{K},\mathbb{H}^{n}_{K}RiemannianThms. 4.1 and 4.2
Table 1 : Comparison of hyperbolic FC layers. An extended table can be found in App. F .
SPD FC LayerGeometriesRequirementIncorporated by Ours
Gyro FC [ 56 ]AIM, LEM & LCMGyrovector✓ ( Sec. G.1 )
Flat FC [ 57 ]LEM & LCMFlat geometry✓ ( Sec. G.2 )
Symmetric FC [ 57 ]AIMInvariant metric Symmetric spaceN/A
OursRiemannian spacesRiemannianN/A
Table 2 : Comparison with the existing SPD FC layers.
MethodsPerspectiveFlexible dimensions
Subspace ppAmbient nnChannel
FRMap + ReOrth [ 36 , Eqs. (2-4)]ONB✗✓✗
PP Scaling [ 54 , Sec. 4.2.2]PP✗✗✗
ONB Scaling [ 55 , Sec. 3.2]ONB✗✗✗
GrTrans [ 55 , Sec. 2.3.2]ONB + PP✗✗✗
GrConvONB + PP✓✓✓
Table 3 : Our GrConv against the existing Grassmannian transformation layers.
MethodMechanismSpaceDisease ( δ=0\delta=0 )Airport ( δ=1\delta=1 )Pubmed ( δ=3.5\delta=3.5 )Cora ( δ=11\delta=11 )
AUC#ParamMiBAUC#ParamMiBAUC#ParamMiBAUC#ParamMiB
Möbius [ 28 ]TangentPKn\mathbb{P}^{n}_{K}76.73 ± 4.8646477.2093.26 ± 0.43480123.8494.95 ± 0.068,2883273.1990.75 ± 0.4723,216176.57
Einstein [ 51 ]TangentKKn\mathbb{K}^{n}_{K}77.34 ± 2.5646477.3292.72 ± 0.07480125.7094.99 ± 0.138,2883273.1989.73 ± 0.2123,216176.57
LorentzTanTangentHKn\mathbb{H}^{n}_{K}76.54 ± 0.8346482.0392.70 ± 0.19480125.4694.99 ± 0.138,2883653.4789.73 ± 0.2123,216326.61
LFC [ 14 ]SpacetimeHKn\mathbb{H}^{n}_{K}78.00 ± 0.6056377.2892.63 ± 0.27580118.0194.22 ± 0.118,8763653.4891.74 ± 0.1224,737326.62
NestFC [ 26 ]Nested projectionHKn\mathbb{H}^{n}_{K}71.30 ± 3.5081178.3583.19 ± 0.90850119.2088.07 ± 0.75258,5143659.2086.08 ± 0.842,076,9311234.25
Table 4 : Comparison of hyperbolic FC layers under the same two-layer HNN, reporting AUC (%), full-model registered parameter elements, and peak allocated memory (MiB). The top 3 AUC results are highlighted with red , blue , and cyan . The worst value for each resource is underlined.
DatasetNumber of horospheres50250
Dim8163281632
DiseaseRResNet [ 38 ]76.0 ± 1.778.0 ± 2.277.4 ± 2.271.5 ± 5.178.1 ± 3.376.5 ± 2.5
Möbius+RResNet74.6 ± 1.974.6 ± 5.775.1 ± 2.174.0 ± 2.771.0 ± 5.273.3 ± 3.4
Poincaré FC+RResNet80.4 ± 0.779.1 ± 1.879.1 ± 1.680.6 ± 0.879.1 ± 0.780.1 ± 1.4
HFC-P+RResNet81.1 ± 0.680.0 ± 0.481.0 ± 0.680.9 ± 0.682.3 ± 0.682.1 ± 0.3
AirportRResNet [ 38 ]93.4 ± 1.192.6 ± 1.193.0 ± 0.293.0 ± 0.493.0 ± 1.689.6 ± 4.7
Table 5 : Comparison of hyperbolic transformations under different settings of Poincaré RResNet.
MethodsRadarHDM05FPHANTU60
SPDNet [ 34 ]93.25 ± 1.1064.57 ± 0.6185.59 ± 0.7266.36 ± 0.72
SPDNetBN [ 8 ]94.85 ± 0.9971.28 ± 0.7989.33 ± 0.4969.38 ± 0.84
RResNet-AIM [ 38 ]95.71 ± 0.3764.95 ± 0.8286.63 ± 0.5570.70 ± 3.81
RResNet-LEM [ 38 ]95.89 ± 0.8670.12 ± 2.4585.07 ± 0.9974.67 ± 2.89
SPDNetLieBN-AIM [ 16 ]95.47 ± 0.9071.83 ± 0.6990.39 ± 0.6673.34 ± 0.40
SPDNetLieBN-LCM [ 16 ]94.80 ± 0.7171.78 ± 0.4486.33 ± 0.4372.54 ± 1.09
Table 6 : Comparison of our SPDNNs against other SPD networks. The ones highlighted with ora are our special cases, while those marked with ∗ are reproduced by us because official code is unavailable.
GeometryMethodRadarHDM05FPHANTU60
AIMGyroSPD++5.09103.5766.35125.05
SPDNN4.84101.8065.42124.41
LEMGyroSPD++0.990.950.667.58
SPDNN0.860.740.635.79
LCMGyroSPD++0.660.700.375.74
SPDNN0.650.590.353.72
Table 7 : The average training time per epoch of our SPDNNs compared with GyroSPD++. A full comparison of efficiency can be found in Sec. I.2.5 .
MethodSubspace dimsAmbient dimsMean ± Std
GrNet [ 36 ]420–>1690.48 ± 0.76
GyroGr-Scaling ∗ [ 55 ]420–>2088.88 ± 1.52
GyroGr ∗ [ 55 ]420–>2090.64 ± 0.57
GrNN-ONB4–>420–>1693.92 ± 0.74
4–>420–>2092.83 ± 0.66
4–>620–>1695.23 ± 0.96
Table 8 : Comparison of GrNNs against other Grassmannian networks on the Radar dataset. Those marked with ∗ are reproduced by us because official code is unavailable.
Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

NotationExplanation
{N,gN}\{\mathcal{N},g^{\mathcal{N}}\}Riemannian manifold N\mathcal{N} with Riemannian metric gNg^{\mathcal{N}}
{M,gM}\{\mathcal{M},g^{\mathcal{M}}\}Riemannian manifold M\mathcal{M} with Riemannian metric gMg^{\mathcal{M}}
EEOrigin of the manifold of interest
TPMT_{P}\mathcal{M}Tangent space at P∈MP\in\mathcal{M}
gp(⋅,⋅)g_{p}(\cdot,\cdot) or ⟨⋅,⋅⟩P\langle\cdot,\cdot\rangle_{P}Riemannian metric at PP
∥⋅∥P\|\cdot\|_{P}The norm induced by ⟨⋅,⋅⟩P\langle\cdot,\cdot\rangle_{P} on TPMT_{P}\mathcal{M}
Appendix
Table 9 : Summary of notation.
OperatorPKn={x∈Rn∣∥x∥2<−1K}\mathbb{P}^{n}_{K}=\left\{x\in\mathbb{R}^{n}\mid\left\|x\right\|^{2}<-\frac{1}{K}\right\}HKn={x∈Rn+1∣∥x∥L2=1K,x1>0}\mathbb{H}^{n}_{K}=\left\{x\in\mathbb{R}^{n+1}\mid\left\|x\right\|_{\mathcal{L}}^{2}=\frac{1}{K},x_{1}>0\right\} , with ∥x∥L2=∑i=2n+1xi2−x12\left\|x\right\|_{\mathcal{L}}^{2}=\sum_{i=2}^{n+1}x_{i}^{2}-x_{1}^{2}
gx(v,w)g_{x}(v,w)(λxK)2⟨v,w⟩(\lambda^{K}_{x})^{2}\left\langle v,w\right\rangle λxK=2(1+K∥x∥2)\lambda^{K}_{x}=\frac{2}{\left(1+K\|x\|^{2}\right)}⟨v,w⟩L=∑i=2n+1viwi−v1w1\left\langle v,w\right\rangle_{\mathcal{L}}=\sum_{i=2}^{n+1}v_{i}w_{i}-v_{1}w_{1}
d⁡(x,y)\operatorname{d}(x,y)2∣K∣tanh⁡−1(∣K∣∥−x⊕My∥)\frac{2}{\sqrt{|K|}}\tanh^{-1}\left(\sqrt{|K|}\|-x\oplus_{\mathrm{M}}y\|\right)1∣K∣cosh⁡−1(K⟨x,y⟩L)\frac{1}{\sqrt{|K|}}\cosh^{-1}\left(K\left\langle x,y\right\rangle_{\mathcal{L}}\right)
Log⁡x(y)\operatorname{Log}_{x}(y)2∣K∣λxKtanh⁡−1(∣K∣∥−x⊕My∥)−x⊕My∥−x⊕My∥\frac{2}{\sqrt{|K|}\lambda_{x}^{K}}\tanh^{-1}\left(\sqrt{|K|}\left\|-x\oplus_{\mathrm{M}}y\right\|\right)\frac{-x\oplus_{\mathrm{M}}y}{\left\|-x\oplus_{\mathrm{M}}y\right\|}cosh⁡−1(K⟨x,y⟩L)sinh⁡(cosh⁡−1(K⟨x,y⟩L))(y−K⟨x,y⟩Lx)\frac{\cosh^{-1}\left(K\langle x,y\rangle_{\mathcal{L}}\right)}{\sinh\left(\cosh^{-1}\left(K\langle x,y\rangle_{\mathcal{L}}\right)\right)}\left(y-K\langle x,y\rangle_{\mathcal{L}}x\right)
Γx→y(v)\Gamma_{x\rightarrow y}(v)λxKλyKgyr⁡[y,−x]v\frac{\lambda_{x}^{K}}{\lambda_{y}^{K}}\operatorname{gyr}[y,-x]vv−K⟨y,v⟩L1+K⟨x,y⟩L(x+y)v-\frac{K\langle y,v\rangle_{\mathcal{L}}}{1+K\langle x,y\rangle_{\mathcal{L}}}(x+y)
Exp⁡x(v)\operatorname{Exp}_{x}(v)x⊕M(tanh⁡(∣K∣λxK∥v∥2)v∣K∣∥v∥)x\oplus_{\mathrm{M}}\left(\tanh\left(\sqrt{|K|}\frac{\lambda_{x}^{K}\|v\|}{2}\right)\frac{v}{\sqrt{|K|}\|v\|}\right)cosh⁡(∣K∣∥v∥L)x+sinh⁡(∣K∣∥v∥L)v∣K∣∥v∥L\cosh\left(\sqrt{|K|}\left\|v\right\|_{\mathcal{L}}\right)x+\sinh\left(\sqrt{|K|}\left\|v\right\|_{\mathcal{L}}\right)\frac{v}{\sqrt{|K|}\left\|v\right\|_{\mathcal{L}}}
References[ 28 , 67 , 72 ][ 60 , 67 ]
Appendix
Table 10 : Riemannian operators on the Poincaré ball and hyperboloid (K<0)(K<0) .
OperatorsKKn={x∈Rn∣∥x∥2<−1K}\mathbb{K}^{n}_{K}=\{x\in\mathbb{R}^{n}\mid\|x\|^{2}<-\tfrac{1}{K}\}
gx(v,w)g_{x}(v,w)⟨v,w⟩1+K∥x∥2−K⟨x,v⟩⟨x,w⟩(1+K∥x∥2)2\frac{\left\langle v,w\right\rangle}{1+K\left\|x\right\|^{2}}-\frac{K\left\langle x,v\right\rangle\left\langle x,w\right\rangle}{\left(1+K\left\|x\right\|^{2}\right)^{2}}
d⁡(x,y)\operatorname{d}(x,y)2−Ktanh⁡−1(−K∥−x⊕Ey∥1+1+K∥−x⊕Ey∥2)\frac{2}{\sqrt{-K}}\tanh^{-1}\left(\sqrt{-K}\frac{\|-x\oplus_{\mathrm{E}}y\|}{1+\sqrt{1+K\|-x\oplus_{\mathrm{E}}y\|^{2}}}\right)
Exp⁡x(v)\operatorname{Exp}_{x}(v)x⊕EExp⁡0(11+K∥x∥2v−K⟨x,v⟩(1+1+K∥x∥2)(1+K∥x∥2)x)x\oplus_{\mathrm{E}}\operatorname{Exp}_{\mathbf{0}}\left(\frac{1}{\sqrt{1+K\|x\|^{2}}}v-\frac{K\langle x,v\rangle}{(1+\sqrt{1+K\|x\|^{2}})(1+K\|x\|^{2})}x\right)
Log⁡x(y)\operatorname{Log}_{x}(y)1λx~K(πPKn→KKn)∗,x~(Log⁡0(−x⊕Ey)),x~=πKKn→PKn(x)\frac{1}{\lambda_{\tilde{x}}^{K}}(\pi_{\mathbb{P}^{n}_{K}\to\mathbb{K}^{n}_{K}})_{*,\tilde{x}}\left(\operatorname{Log}_{\mathbf{0}}(-x\oplus_{\mathrm{E}}y)\right),\quad\tilde{x}=\pi_{\mathbb{K}^{n}_{K}\to\mathbb{P}^{n}_{K}}(x)
References[ 73 , 20 ]
Appendix
Table 11 : Riemannian operators on the Beltrami–Klein model (K<0)(K<0) .
OperatorsLEMAIMPEM
gP(V,W)g_{P}(V,W)⟨log⁡∗,P(V),log⁡∗,P(W)⟩(α,β)\langle\log_{*,P}(V),\log_{*,P}(W)\rangle^{(\alpha,\beta)}⟨P−1V,WP−1⟩(α,β)\langle P^{-1}V,WP^{-1}\rangle^{(\alpha,\beta)}1θ2⟨P⁡θ∗,P(V),P⁡θ∗,P(W)⟩(α,β)\frac{1}{\theta^{2}}\langle\operatorname{P}_{\theta*,P}(V),\operatorname{P}_{\theta*,P}(W)\rangle^{(\alpha,\beta)}
Log⁡PQ\operatorname{Log}_{P}Q(log⁡∗,P)−1[log⁡(Q)−log⁡(P)](\log_{*,P})^{-1}\left[\log(Q)-\log(P)\right]P12log⁡(P−12QP−12)P12P^{\frac{1}{2}}\log\left(P^{-\frac{1}{2}}QP^{-\frac{1}{2}}\right)P^{\frac{1}{2}}(Pθ∗,P)−1(Qθ−Pθ)(P_{\theta*,P})^{-1}\left(Q^{\theta}-P^{\theta}\right)
ΓP→Q(V)\Gamma_{P\rightarrow Q}(V)(log⁡∗,Q)−1∘log⁡∗,P(V)(\log_{*,Q})^{-1}\circ\log_{*,P}(V)(QP−1)12V(P−1Q)12(QP^{-1})^{\frac{1}{2}}V(P^{-1}Q)^{\frac{1}{2}}(P⁡θ∗,Q)−1∘P⁡θ∗,P(V)(\operatorname{P}_{\theta*,Q})^{-1}\circ\operatorname{P}_{\theta*,P}(V)
Exp⁡P(V)\operatorname{Exp}_{P}(V)exp⁡(log⁡(P)+log⁡∗,P(V))\exp\left(\log(P)+\log_{*,P}(V)\right)P12exp⁡(P−12VP−12)P12P^{\frac{1}{2}}\exp\left(P^{-\frac{1}{2}}VP^{-\frac{1}{2}}\right)P^{\frac{1}{2}}(Pθ+Pθ∗,P(V))1θ\left(P^{\theta}+P_{\theta*,P}(V)\right)^{\frac{1}{\theta}}
InvarianceLie group bi-invariance O(n)\mathrm{O}({n}) -invarianceLie group left-invariance GL(n)\mathrm{GL}({n}) -invarianceO(n)\mathrm{O}({n}) -invariance
References[ 2 , 70 ][ 59 , 68 ][ 24 , 70 , 17 ]
Appendix
Table 12 : The Riemannian operators under LEM, AIM, and PEM on the SPD manifold.
OperatorsLCMBWM
gP(V,W)g_{P}(V,W)⟨⌊V~⌋,⌊W~⌋⟩+⟨V~L~−1,W~L~−1⟩\langle\lfloor\widetilde{V}\rfloor,\lfloor\widetilde{W}\rfloor\rangle+\langle\widetilde{\mathbb{V}}\widetilde{\mathbb{L}}^{-1},\widetilde{\mathbb{W}}\widetilde{\mathbb{L}}^{-1}\rangle12⟨LP[V],W⟩\frac{1}{2}\langle\mathcal{L}_{P}[V],W\rangle
Log⁡PQ\operatorname{Log}_{P}Q(Chol⁡−1)∗,L[⌊K⌋−⌊L⌋+LDlog⁡(L−1K)](\operatorname{Chol}^{-1})_{*,L}\left[\lfloor K\rfloor-\lfloor L\rfloor+\mathbb{L}\operatorname{Dlog}(\mathbb{L}^{-1}\mathbb{K})\right](PQ)12+(QP)12−2P(PQ)^{\frac{1}{2}}+(QP)^{\frac{1}{2}}-2P
ΓP→Q(V)\Gamma_{P\rightarrow Q}(V)(Chol⁡−1)∗,K[⌊V~⌋+KL−1V~](\operatorname{Chol}^{-1})_{*,K}\left[\lfloor\widetilde{V}\rfloor+\mathbb{K}\mathbb{L}^{-1}\widetilde{\mathbb{V}}\right]U[δi+δjσi+σj[U⊤VU]ij]U⊤U\left[\sqrt{\frac{\delta_{i}+\delta_{j}}{\sigma_{i}+\sigma_{j}}}\left[U^{\top}VU\right]_{ij}\right]U^{\top}
Exp⁡P(V)\operatorname{Exp}_{P}(V)Chol⁡−1[⌊L⌋+⌊V~⌋+LDexp⁡(L−1V~)]\operatorname{Chol}^{-1}\left[\lfloor L\rfloor+\lfloor\widetilde{V}\rfloor+\mathbb{L}\operatorname{Dexp}(\mathbb{L}^{-1}\widetilde{V})\right]P+V+LP[V]PLP[V]P+V+\mathcal{L}_{P}[V]P\mathcal{L}_{P}[V]
InvarianceLie group bi-invarianceO(n)\mathrm{O}({n}) -invariance
References[ 46 ][ 6 , 70 ]
Appendix
Table 13 : The Riemannian operators under BWM and LCM on the SPD manifold.
OperatorsGr(p,n)\mathrm{Gr}(p,n)Gr~(p,n)\widetilde{\mathrm{Gr}}(p,n)
gP(V,W)g_{P}(V,W)⟨V,W⟩\left\langle V,W\right\rangle12⟨V,W⟩\frac{1}{2}\left\langle V,W\right\rangle
Log⁡PQ\operatorname{Log}_{P}QOarctan⁡(Σ)R⊤O\arctan(\Sigma)R^{\top} (In−PP⊤)Q(P⊤Q)−1:=SVDOΣR⊤(I_{n}-PP^{\top})Q(P^{\top}Q)^{-1}\stackrel{{\scriptstyle\mathrm{SVD}}}{{:=}}O\Sigma R^{\top}12[log⁡((In−2Q)(In−2P)),P]\frac{1}{2}[\log\left(\left(I_{n}-2Q\right)\left(I_{n}-2P\right)\right),P]
ΓP→Q(V)\Gamma_{P\rightarrow Q}(V)\left(\left(\begin{array}[]{cc}PR&O\end{array}\right)\left(\begin{array}[]{c}-\sin(\Sigma)\\ \cos(\Sigma)\end{array}\right)O^{T}+\left(I-OO^{T}\right)\right)V Log⁡P(Q):=SVDOΣR⊤\operatorname{Log}_{P}(Q)\stackrel{{\scriptstyle\mathrm{SVD}}}{{:=}}O\Sigma R^{\top}exp⁡([log⁡P(Q),P])Vexp⁡(−[log⁡P(Q),P])\exp(\left[\log_{P}(Q),P\right])V\exp(-[\log_{P}(Q),P])
Exp⁡PV\operatorname{Exp}_{P}V\left(\begin{array}[]{cc}PR&O\end{array}\right)\left(\begin{array}[]{c}\cos(\Sigma)\\ \sin(\Sigma)\end{array}\right)R^{\top} V:=SVDOΣR⊤V\stackrel{{\scriptstyle\mathrm{SVD}}}{{:=}}O\Sigma R^{\top}exp⁡([V,P])Pexp⁡(−[V,P])\exp([V,P])P\exp(-[V,P])
References[ 25 , 5 ][ 3 , 5 ]
Appendix
Table 14 : Riemannian operators on the Grassmannian.
MethodModelMechanismFormulationParametersReferences
MöbiusPKn\mathbb{P}^{n}_{K}TangentExp⁡0(MLog⁡0(x))\operatorname{Exp}_{\mathbf{0}}(M\operatorname{Log}_{\mathbf{0}}(x))M∈Rm×nM\in\mathbb{R}^{m\times n}[ 28 , Def. 3.2]
KleinKKn\mathbb{K}^{n}_{K}TangentExp⁡0(MLog⁡0(x))\operatorname{Exp}_{\mathbf{0}}(M\operatorname{Log}_{\mathbf{0}}(x))M∈Rm×nM\in\mathbb{R}^{m\times n}[ 51 , Thm. 9]
LFCHKn\mathbb{H}^{n}_{K}Spacetime\displaystyle\left[\begin{array}[]{c}\frac{\sqrt{\|Wx\|^{2}-1/K}}{v^{\top}x}v^{\top}\\ W\end{array}\right]xM∈Rm×(n+1)M\in\mathbb{R}^{m\times(n+1)} v∈Rn+1v\in\mathbb{R}^{n+1}[ 14 , Sec 3.1]
NestFCHKn\mathbb{H}^{n}_{K}Nested projectiony=Wx∥Wx∥L,WJnW⊤=Jm\displaystyle y=\frac{Wx}{\|Wx\|_{L}},\quad WJ_{n}W^{\top}=J_{m}Q∈SO(n)Q\in\mathrm{SO}(n) P~∈St(m,n)\widetilde{P}\in\mathrm{St}(m,n) α∈R\alpha\in\mathbb{R}[ 26 , Eq. (14) and Sec. 3.3]
Poincaré FCPKn\mathbb{P}^{n}_{K}Poincaréw(1+1−K∥w∥2)−1w\left(1+\sqrt{1-K\|w\|^{2}}\right)^{-1} w=((−K)−12sinh⁡(−Kvk(x)))k=1mw=\left((-K)^{-\frac{1}{2}}\sinh\left(\sqrt{-K}v_{k}(x)\right)\right)_{k=1}^{m} vkv_{k} is defined by Shimizu et al. [66, Eq. (6)]{zi∈Rn}i=1m\{z_{i}\in\mathbb{R}^{n}\}_{i=1}^{m} {γi∈R}i=1m\{\gamma_{i}\in\mathbb{R}\}_{i=1}^{m}[ 66 , Sec. 3.2]
OursPKn,KKn,HKn\mathbb{P}^{n}_{K},\mathbb{K}^{n}_{K},\mathbb{H}^{n}_{K}RiemannianThms. 4.1 and 4.2{zi∈Rn}i=1m\{z_{i}\in\mathbb{R}^{n}\}_{i=1}^{m} {γi∈R}i=1m\{\gamma_{i}\in\mathbb{R}\}_{i=1}^{m}Thms. 4.1 and 4.2
Appendix
Table 15 : Comparison of hyperbolic linear layers. Here, we consider the transformation from an nn -dimensional hyperbolic space to an mm -dimensional one.
MethodSpaceComplexityParameter dimension
Möbius [ 28 ]PKn\mathbb{P}^{n}_{K}O(Bnm)O(Bnm)nmnm
Einstein [ 51 ]KKn\mathbb{K}^{n}_{K}O(Bnm)O(Bnm)nmnm
LorentzTanHKn\mathbb{H}^{n}_{K}O(Bnm)O(Bnm)nmnm
LFC [ 14 ]HKn\mathbb{H}^{n}_{K}O(Bnm)O(Bnm)m(n+1)+m+(n+1)+2m(n+1)+m+(n+1)+2
NestFC [ 26 ]HKn\mathbb{H}^{n}_{K}O(n3+Bn2)‾\underline{O(n^{3}+Bn^{2})}n(n−1)2+mn−m(m+1)2+1‾\underline{\frac{n(n-1)}{2}+mn-\frac{m(m+1)}{2}+1}
Poincaré FC [ 66 ]PKn\mathbb{P}^{n}_{K}O(Bnm)O(Bnm)nm+mnm+m
Appendix
Table 17 : Forward complexity and parameter dimensions of an nn -to- mm hyperbolic FC layer for a batch of BB samples in the reduction setting m≤nm\leq n . The worst entry in each metric is underlined.
StepHyperparameterCandidates
1Feature normalization ( normalize_feats )0,10,1
2ActivationNone, ReLU, LeakyReLU, ELU, Tanh
3Learning rate0.001,0.005,0.008,0.01,0.020.001,0.005,0.008,0.01,0.02
4Gyro bias0,10,1 for HFC-H only. NestFC skips this step
5Gradient clippingNone, 0.5,1.00.5,1.0
6Attention and local aggregation(0,0),(1,1)(0,0),(1,1) for ( use_att , local_agg )
Appendix
Table 18 : Hyperparameter-search procedure.
MethodDiseaseAirportPubmedCora
NHGCN-NestFC78.28 ± 0.7394.49 ± 0.0494.70 ± 0.2193.11 ± 0.40
NHGCN-HFC-H97.37 ± 0.2296.14 ± 0.2796.18 ± 0.0692.85 ± 0.24
Appendix
Table 19 : Link-prediction test AUC (%).
MethodDiseaseAirportPubmedCora
#ParamMiBFitTime#ParamMiBFitTime#ParamMiBFitTime#ParamMiBFitTime
NHGCN-NestFC73881.1027.039776120.2943.292257,9523496.40105.0942,075,4361266.5194.458
NHGCN-HFC-H45092.8819.854465134.3444.6387,7853444.3291.06421,780253.6824.007
Appendix
Table 20 : Full-model trainable parameters, peak memory (MiB), and FitTime (ms/epoch) for NHGCN-NestFC and NHGCN-HFC-H.
DatasetModelθ\thetaOptimizerLearning Rate
RadarSPDNN-LEMN/AAMSGrad5e−35e^{-3}
SPDNN-AIM0.25AMSGrad5e−45e^{-4}
SPDNN-PEMN/AAMSGrad1e−21e^{-2}
SPDNN-LCM0.25AMSGrad5e−45e^{-4}
SPDNN-BWMN/AAMSGrad5e−45e^{-4}
HDM05SPDNN-LEMN/ASGD5e−35e^{-3}
Appendix
Table 21 : Training hyperparameters in SPDNNs.
MethodFPHA
SourceOursGap
SPDNet (official code)88.79±0.3688.79\pm 0.3685.59±0.7285.59\pm 0.72−3.20-3.20
SPDNetBN (official code)91.02±0.2591.02\pm 0.2589.33±0.4989.33\pm 0.49−1.69-1.69
GyroLE94.6194.6190.73±0.9290.73\pm 0.92−3.88-3.88
GyroLC82.4382.4376.10±0.6376.10\pm 0.63−6.33-6.33
GyroAI93.3993.3989.60±0.3789.60\pm 0.37−3.79-3.79
Appendix
Table 22 : FPHA accuracy (%): Source [ 55 , 56 ] , Ours, and Gap (Ours −- Source; percentage points).
MethodGeometryRadarHDM05FPHANTU60
SPDNetN/A0.660.500.283.08
SPDNetBNAIM1.250.940.586.14
SPDResNet-AIMAIM0.961.230.696.84
SPDResNet-LEMLEM0.770.550.303.17
SPDNetLieBN-AIMAIM1.211.150.978.85
SPDNetLieBN-LCMLCM1.101.110.595.96
Appendix
Table 23 : Training efficiency (seconds per epoch).
Input shape#ConvKernelsKernel sizesStridesShape changesFinal SPD features
[40,3,3][40,3,3]3[4,8,16][4,8,16][3,3,3][3,3,3][2,2,2][2,2,2][40,3,3]→[19,4,3,3]→[9,8,3,3]→[4,16,3,3][40,3,3]\to[19,4,3,3]\to[9,8,3,3]\to[4,16,3,3][64,3,3][64,3,3]
[20,6,6][20,6,6]3[4,8,16][4,8,16][2,2,2][2,2,2][2,2,1][2,2,1][20,6,6]→[10,4,6,6]→[5,8,6,6]→[4,16,6,6][20,6,6]\to[10,4,6,6]\to[5,8,6,6]\to[4,16,6,6][64,6,6][64,6,6]
[5,12,12][5,12,12]2[8,20][8,20][2,2][2,2][1,1][1,1][5,12,12]→[4,8,12,12]→[3,20,12,12][5,12,12]\to[4,8,12,12]\to[3,20,12,12][60,12,12][60,12,12]
Appendix
Table 24 : SPD CNN architectures with local receptive fields and shared kernels.
Method[40,3,3][40,3,3][20,6,6][20,6,6][5,12,12][5,12,12]
AccuracyFitTimeAccuracyFitTimeAccuracyFitTime
ManifoldNet77.89 ± 1.5613.1273.27 ± 0.6811.5880.75 ± 1.2311.50
MVC-Net81.04 ± 1.727.8878.05 ± 0.238.1881.15 ± 0.347.87
SPDConvNet-LEM97.68 ± 0.321.5895.86 ± 0.951.5596.91 ± 0.181.031.03
SPDConvNet-LCM96.51 ± 0.661.541.5494.40 ± 0.501.221.2297.20 ± 0.651.25
SPDConvNet-PEM97.44 ± 0.561.8294.45 ± 0.781.5396.16 ± 0.912.34
Appendix
Table 25 : Radar classification accuracy (%) and FitTime (s/epoch) under different architectures.
Learning rate5×10−55\times 10^{-5}10−410^{-4}2×10−42\times 10^{-4}5×10−45\times 10^{-4}10−310^{-3}2×10−32\times 10^{-3}5×10−35\times 10^{-3}10−210^{-2}
Accuracy (%)55.20 ± 2.5859.73 ± 1.3163.52 ± 1.4469.55 ± 2.4673.28 ± 1.4276.08 ± 1.0877.89 ± 1.5677.12 ± 1.30
Appendix
Table 26 : ManifoldNet learning-rate ablation under the [40,3,3][40,3,3] input setting.

Explore similar work

CardsList

  1. Riemannian Deep Learning: Modules, Networks, and Geometries

    Jul 21, 2026Ziheng ChenData ManifoldGeometric Deep Learning

    Doctoral School in Information and Communication Technology, Università di Trento

  2. Batch Normalization for Neural Networks on Complex Domains

    May 1, 2026Xuan Son Nguyen, Nistor GrozavuBatch NormalizationNeural Network

    ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, Cergy, France

  3. Riemannian Networks over Full-Rank Correlation Matrices

    May 18, 2026Ziheng Chen, Xiaojun Wu, Bernhard Schölkopf +1Full-Rank Correlation MatricesHigh-Order Correlations

    Department of Information Engineering and Computer Science, University of Trento, Trento, Italy · Max Planck Institute for Intelligent Systems, Tübingen, Germany · School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China