Building Transformation Layers for Riemannian Neural Networks

Ziheng Chen

cs.AISep 28, 2026

Building Transformation Layers for Riemannian Neural Networks

Authors: Ziheng Chen

Organizations: University of Trento · MPI for Intelligent Systems, Tübingen

Abstract

Recently, deep neural networks on manifold-valued representations have garnered significant attention across various machine learning applications. One recent focus is the generalization of Euclidean fully connected (FC) and convolutional layers to non-Euclidean geometries. However, previous approaches typically focus on a few selected manifolds and rely on specific properties of the target manifold. In contrast, this work proposes a framework for constructing FC and convolutional layers over computationally tractable Riemannian spaces. This framework incorporates several previous FC layers across different geometries as special cases and is instantiated on ten representative manifolds, including three hyperbolic models, five geometries of the symmetric positive definite (SPD) manifold, and two Grassmannian perspectives. Experiments on different manifolds demonstrate the effectiveness and applicability of our approach. Code can be found at https://github.com/GitZH-Chen/RieTrans.

Figures & tables

Figure 1 : Riemannian convolution within a receptive field. Here, $\mathcal{F}^{k}(\cdot)$ denotes the $k$ -th FC transformation.

Method	Space	Mechanism	References
Möbius	$\mathbb{P}^{n}_{K}$	Tangent	[ 28 , Def. 3.2]
Einstein	$\mathbb{K}^{n}_{K}$	Tangent	[ 51 , Thm. 9]
LFC	$\mathbb{H}^{n}_{K}$	Spacetime	[ 14 , Eq. (1)]
NestFC	$\mathbb{H}^{n}_{K}$	Nested projection	[ 26 , Eq. (14)]
Poincaré FC	$\mathbb{P}^{n}_{K}$	Poincaré	[ 66 , Sec. 3.2]
Ours	$\mathbb{P}^{n}_{K},\mathbb{K}^{n}_{K},\mathbb{H}^{n}_{K}$	Riemannian	Thms. 4.1 and 4.2

Table 1 : Comparison of hyperbolic FC layers. An extended table can be found in App. F .

SPD FC Layer	Geometries	Requirement	Incorporated by Ours
Gyro FC [ 56 ]	AIM, LEM & LCM	Gyrovector	✓ ( Sec. G.1 )
Flat FC [ 57 ]	LEM & LCM	Flat geometry	✓ ( Sec. G.2 )
Symmetric FC [ 57 ]	AIM	Invariant metric Symmetric space	N/A
Ours	Riemannian spaces	Riemannian	N/A

Table 2 : Comparison with the existing SPD FC layers.

Methods	Perspective	Flexible dimensions
Methods	Perspective	Subspace $p$	Ambient $n$	Channel
FRMap + ReOrth [ 36 , Eqs. (2-4)]	ONB	✗	✓	✗
PP Scaling [ 54 , Sec. 4.2.2]	PP	✗	✗	✗
ONB Scaling [ 55 , Sec. 3.2]	ONB	✗	✗	✗
GrTrans [ 55 , Sec. 2.3.2]	ONB + PP	✗	✗	✗
GrConv	ONB + PP	✓	✓	✓

Table 3 : Our GrConv against the existing Grassmannian transformation layers.

Method	Mechanism	Space	Disease ( $\delta=0$ )			Airport ( $\delta=1$ )			Pubmed ( $\delta=3.5$ )			Cora ( $\delta=11$ )
Method	Mechanism	Space	AUC	#Param	MiB	AUC	#Param	MiB	AUC	#Param	MiB	AUC	#Param	MiB
Möbius [ 28 ]	Tangent	$\mathbb{P}^{n}_{K}$	76.73 ± 4.86	464	77.20	93.26 ± 0.43	480	123.84	94.95 ± 0.06	8,288	3273.19	90.75 ± 0.47	23,216	176.57
Einstein [ 51 ]	Tangent	$\mathbb{K}^{n}_{K}$	77.34 ± 2.56	464	77.32	92.72 ± 0.07	480	125.70	94.99 ± 0.13	8,288	3273.19	89.73 ± 0.21	23,216	176.57
LorentzTan	Tangent	$\mathbb{H}^{n}_{K}$	76.54 ± 0.83	464	82.03	92.70 ± 0.19	480	125.46	94.99 ± 0.13	8,288	3653.47	89.73 ± 0.21	23,216	326.61
LFC [ 14 ]	Spacetime	$\mathbb{H}^{n}_{K}$	78.00 ± 0.60	563	77.28	92.63 ± 0.27	580	118.01	94.22 ± 0.11	8,876	3653.48	91.74 ± 0.12	24,737	326.62
NestFC [ 26 ]	Nested projection	$\mathbb{H}^{n}_{K}$	71.30 ± 3.50	811	78.35	83.19 ± 0.90	850	119.20	88.07 ± 0.75	258,514	3659.20	86.08 ± 0.84	2,076,931	1234.25

Table 4 : Comparison of hyperbolic FC layers under the same two-layer HNN, reporting AUC (%), full-model registered parameter elements, and peak allocated memory (MiB). The top 3 AUC results are highlighted with red , blue , and cyan . The worst value for each resource is underlined.

Dataset	Number of horospheres	50			250
Dataset	Dim	8	16	32	8	16	32
Disease	RResNet [ 38 ]	76.0 ± 1.7	78.0 ± 2.2	77.4 ± 2.2	71.5 ± 5.1	78.1 ± 3.3	76.5 ± 2.5
	Möbius+RResNet	74.6 ± 1.9	74.6 ± 5.7	75.1 ± 2.1	74.0 ± 2.7	71.0 ± 5.2	73.3 ± 3.4
	Poincaré FC+RResNet	80.4 ± 0.7	79.1 ± 1.8	79.1 ± 1.6	80.6 ± 0.8	79.1 ± 0.7	80.1 ± 1.4
	HFC-P+RResNet	81.1 ± 0.6	80.0 ± 0.4	81.0 ± 0.6	80.9 ± 0.6	82.3 ± 0.6	82.1 ± 0.3
Airport	RResNet [ 38 ]	93.4 ± 1.1	92.6 ± 1.1	93.0 ± 0.2	93.0 ± 0.4	93.0 ± 1.6	89.6 ± 4.7

Table 5 : Comparison of hyperbolic transformations under different settings of Poincaré RResNet.

Methods	Radar	HDM05	FPHA	NTU60
SPDNet [ 34 ]	93.25 ± 1.10	64.57 ± 0.61	85.59 ± 0.72	66.36 ± 0.72
SPDNetBN [ 8 ]	94.85 ± 0.99	71.28 ± 0.79	89.33 ± 0.49	69.38 ± 0.84
RResNet-AIM [ 38 ]	95.71 ± 0.37	64.95 ± 0.82	86.63 ± 0.55	70.70 ± 3.81
RResNet-LEM [ 38 ]	95.89 ± 0.86	70.12 ± 2.45	85.07 ± 0.99	74.67 ± 2.89
SPDNetLieBN-AIM [ 16 ]	95.47 ± 0.90	71.83 ± 0.69	90.39 ± 0.66	73.34 ± 0.40
SPDNetLieBN-LCM [ 16 ]	94.80 ± 0.71	71.78 ± 0.44	86.33 ± 0.43	72.54 ± 1.09

Table 6 : Comparison of our SPDNNs against other SPD networks. The ones highlighted with ora are our special cases, while those marked with ∗ are reproduced by us because official code is unavailable.

Geometry	Method	Radar	HDM05	FPHA	NTU60
AIM	GyroSPD++	5.09	103.57	66.35	125.05
AIM	SPDNN	4.84	101.80	65.42	124.41
LEM	GyroSPD++	0.99	0.95	0.66	7.58
LEM	SPDNN	0.86	0.74	0.63	5.79
LCM	GyroSPD++	0.66	0.70	0.37	5.74
LCM	SPDNN	0.65	0.59	0.35	3.72

Table 7 : The average training time per epoch of our SPDNNs compared with GyroSPD++. A full comparison of efficiency can be found in Sec. I.2.5 .

Method	Subspace dims	Ambient dims	Mean ± Std
GrNet [ 36 ]	4	20–>16	90.48 ± 0.76
GyroGr-Scaling ∗ [ 55 ]	4	20–>20	88.88 ± 1.52
GyroGr ∗ [ 55 ]	4	20–>20	90.64 ± 0.57
GrNN-ONB	4–>4	20–>16	93.92 ± 0.74
	4–>4	20–>20	92.83 ± 0.66
	4–>6	20–>16	95.23 ± 0.96

Table 8 : Comparison of GrNNs against other Grassmannian networks on the Radar dataset. Those marked with ∗ are reproduced by us because official code is unavailable.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Explanation
$\{\mathcal{N},g^{\mathcal{N}}\}$	Riemannian manifold $\mathcal{N}$ with Riemannian metric $g^{\mathcal{N}}$
$\{\mathcal{M},g^{\mathcal{M}}\}$	Riemannian manifold $\mathcal{M}$ with Riemannian metric $g^{\mathcal{M}}$
$E$	Origin of the manifold of interest
$T_{P}\mathcal{M}$	Tangent space at $P\in\mathcal{M}$
$g_{p}(\cdot,\cdot)$ or $\langle\cdot,\cdot\rangle_{P}$	Riemannian metric at $P$
$\\|\cdot\\|_{P}$	The norm induced by $\langle\cdot,\cdot\rangle_{P}$ on $T_{P}\mathcal{M}$

Appendix

Table 9 : Summary of notation.

Operator	$\mathbb{P}^{n}_{K}=\left\{x\in\mathbb{R}^{n}\mid\left\\|x\right\\|^{2}<-\frac{1}{K}\right\}$	$\mathbb{H}^{n}_{K}=\left\{x\in\mathbb{R}^{n+1}\mid\left\\|x\right\\|_{\mathcal{L}}^{2}=\frac{1}{K},x_{1}>0\right\}$ , with $\left\\|x\right\\|_{\mathcal{L}}^{2}=\sum_{i=2}^{n+1}x_{i}^{2}-x_{1}^{2}$
$g_{x}(v,w)$	$(\lambda^{K}_{x})^{2}\left\langle v,w\right\rangle$ $\lambda^{K}_{x}=\frac{2}{\left(1+K\\|x\\|^{2}\right)}$	$\left\langle v,w\right\rangle_{\mathcal{L}}=\sum_{i=2}^{n+1}v_{i}w_{i}-v_{1}w_{1}$
$\operatorname{d}(x,y)$	$\frac{2}{\sqrt{\|K\|}}\tanh^{-1}\left(\sqrt{\|K\|}\\|-x\oplus_{\mathrm{M}}y\\|\right)$	$\frac{1}{\sqrt{\|K\|}}\cosh^{-1}\left(K\left\langle x,y\right\rangle_{\mathcal{L}}\right)$
$\operatorname{Log}_{x}(y)$	$\frac{2}{\sqrt{\|K\|}\lambda_{x}^{K}}\tanh^{-1}\left(\sqrt{\|K\|}\left\\|-x\oplus_{\mathrm{M}}y\right\\|\right)\frac{-x\oplus_{\mathrm{M}}y}{\left\\|-x\oplus_{\mathrm{M}}y\right\\|}$	$\frac{\cosh^{-1}\left(K\langle x,y\rangle_{\mathcal{L}}\right)}{\sinh\left(\cosh^{-1}\left(K\langle x,y\rangle_{\mathcal{L}}\right)\right)}\left(y-K\langle x,y\rangle_{\mathcal{L}}x\right)$
$\Gamma_{x\rightarrow y}(v)$	$\frac{\lambda_{x}^{K}}{\lambda_{y}^{K}}\operatorname{gyr}[y,-x]v$	$v-\frac{K\langle y,v\rangle_{\mathcal{L}}}{1+K\langle x,y\rangle_{\mathcal{L}}}(x+y)$
$\operatorname{Exp}_{x}(v)$	$x\oplus_{\mathrm{M}}\left(\tanh\left(\sqrt{\|K\|}\frac{\lambda_{x}^{K}\\|v\\|}{2}\right)\frac{v}{\sqrt{\|K\|}\\|v\\|}\right)$	$\cosh\left(\sqrt{\|K\|}\left\\|v\right\\|_{\mathcal{L}}\right)x+\sinh\left(\sqrt{\|K\|}\left\\|v\right\\|_{\mathcal{L}}\right)\frac{v}{\sqrt{\|K\|}\left\\|v\right\\|_{\mathcal{L}}}$
References	[ 28 , 67 , 72 ]	[ 60 , 67 ]

Appendix

Table 10 : Riemannian operators on the Poincaré ball and hyperboloid

(K<0)

Operators	$\mathbb{K}^{n}_{K}=\{x\in\mathbb{R}^{n}\mid\\|x\\|^{2}<-\tfrac{1}{K}\}$
$g_{x}(v,w)$	$\frac{\left\langle v,w\right\rangle}{1+K\left\\|x\right\\|^{2}}-\frac{K\left\langle x,v\right\rangle\left\langle x,w\right\rangle}{\left(1+K\left\\|x\right\\|^{2}\right)^{2}}$
$\operatorname{d}(x,y)$	$\frac{2}{\sqrt{-K}}\tanh^{-1}\left(\sqrt{-K}\frac{\\|-x\oplus_{\mathrm{E}}y\\|}{1+\sqrt{1+K\\|-x\oplus_{\mathrm{E}}y\\|^{2}}}\right)$
$\operatorname{Exp}_{x}(v)$	$x\oplus_{\mathrm{E}}\operatorname{Exp}_{\mathbf{0}}\left(\frac{1}{\sqrt{1+K\\|x\\|^{2}}}v-\frac{K\langle x,v\rangle}{(1+\sqrt{1+K\\|x\\|^{2}})(1+K\\|x\\|^{2})}x\right)$
$\operatorname{Log}_{x}(y)$	$\frac{1}{\lambda_{\tilde{x}}^{K}}(\pi_{\mathbb{P}^{n}_{K}\to\mathbb{K}^{n}_{K}})_{*,\tilde{x}}\left(\operatorname{Log}_{\mathbf{0}}(-x\oplus_{\mathrm{E}}y)\right),\quad\tilde{x}=\pi_{\mathbb{K}^{n}_{K}\to\mathbb{P}^{n}_{K}}(x)$
References	[ 73 , 20 ]

Appendix

Table 11 : Riemannian operators on the Beltrami–Klein model

(K<0)

Operators	LEM	AIM	PEM
$g_{P}(V,W)$	$\langle\log_{,P}(V),\log_{,P}(W)\rangle^{(\alpha,\beta)}$	$\langle P^{-1}V,WP^{-1}\rangle^{(\alpha,\beta)}$	$\frac{1}{\theta^{2}}\langle\operatorname{P}_{\theta,P}(V),\operatorname{P}_{\theta,P}(W)\rangle^{(\alpha,\beta)}$
$\operatorname{Log}_{P}Q$	$(\log_{*,P})^{-1}\left[\log(Q)-\log(P)\right]$	$P^{\frac{1}{2}}\log\left(P^{-\frac{1}{2}}QP^{-\frac{1}{2}}\right)P^{\frac{1}{2}}$	$(P_{\theta*,P})^{-1}\left(Q^{\theta}-P^{\theta}\right)$
$\Gamma_{P\rightarrow Q}(V)$	$(\log_{,Q})^{-1}\circ\log_{,P}(V)$	$(QP^{-1})^{\frac{1}{2}}V(P^{-1}Q)^{\frac{1}{2}}$	$(\operatorname{P}_{\theta,Q})^{-1}\circ\operatorname{P}_{\theta,P}(V)$
$\operatorname{Exp}_{P}(V)$	$\exp\left(\log(P)+\log_{*,P}(V)\right)$	$P^{\frac{1}{2}}\exp\left(P^{-\frac{1}{2}}VP^{-\frac{1}{2}}\right)P^{\frac{1}{2}}$	$\left(P^{\theta}+P_{\theta*,P}(V)\right)^{\frac{1}{\theta}}$
Invariance	Lie group bi-invariance $\mathrm{O}({n})$ -invariance	Lie group left-invariance $\mathrm{GL}({n})$ -invariance	$\mathrm{O}({n})$ -invariance
References	[ 2 , 70 ]	[ 59 , 68 ]	[ 24 , 70 , 17 ]

Appendix

Table 12 : The Riemannian operators under LEM, AIM, and PEM on the SPD manifold.

Operators	LCM	BWM
$g_{P}(V,W)$	$\langle\lfloor\widetilde{V}\rfloor,\lfloor\widetilde{W}\rfloor\rangle+\langle\widetilde{\mathbb{V}}\widetilde{\mathbb{L}}^{-1},\widetilde{\mathbb{W}}\widetilde{\mathbb{L}}^{-1}\rangle$	$\frac{1}{2}\langle\mathcal{L}_{P}[V],W\rangle$
$\operatorname{Log}_{P}Q$	$(\operatorname{Chol}^{-1})_{*,L}\left[\lfloor K\rfloor-\lfloor L\rfloor+\mathbb{L}\operatorname{Dlog}(\mathbb{L}^{-1}\mathbb{K})\right]$	$(PQ)^{\frac{1}{2}}+(QP)^{\frac{1}{2}}-2P$
$\Gamma_{P\rightarrow Q}(V)$	$(\operatorname{Chol}^{-1})_{*,K}\left[\lfloor\widetilde{V}\rfloor+\mathbb{K}\mathbb{L}^{-1}\widetilde{\mathbb{V}}\right]$	$U\left[\sqrt{\frac{\delta_{i}+\delta_{j}}{\sigma_{i}+\sigma_{j}}}\left[U^{\top}VU\right]_{ij}\right]U^{\top}$
$\operatorname{Exp}_{P}(V)$	$\operatorname{Chol}^{-1}\left[\lfloor L\rfloor+\lfloor\widetilde{V}\rfloor+\mathbb{L}\operatorname{Dexp}(\mathbb{L}^{-1}\widetilde{V})\right]$	$P+V+\mathcal{L}_{P}[V]P\mathcal{L}_{P}[V]$
Invariance	Lie group bi-invariance	$\mathrm{O}({n})$ -invariance
References	[ 46 ]	[ 6 , 70 ]

Appendix

Table 13 : The Riemannian operators under BWM and LCM on the SPD manifold.

Operators	$\mathrm{Gr}(p,n)$	$\widetilde{\mathrm{Gr}}(p,n)$
$g_{P}(V,W)$	$\left\langle V,W\right\rangle$	$\frac{1}{2}\left\langle V,W\right\rangle$
$\operatorname{Log}_{P}Q$	$O\arctan(\Sigma)R^{\top}$ $(I_{n}-PP^{\top})Q(P^{\top}Q)^{-1}\stackrel{{\scriptstyle\mathrm{SVD}}}{{:=}}O\Sigma R^{\top}$	$\frac{1}{2}[\log\left(\left(I_{n}-2Q\right)\left(I_{n}-2P\right)\right),P]$
$\Gamma_{P\rightarrow Q}(V)$	$\left(\left(\begin{array}[]{cc}PR&O\end{array}\right)\left(\begin{array}[]{c}-\sin(\Sigma)\\ \cos(\Sigma)\end{array}\right)O^{T}+\left(I-OO^{T}\right)\right)V$ $\operatorname{Log}_{P}(Q)\stackrel{{\scriptstyle\mathrm{SVD}}}{{:=}}O\Sigma R^{\top}$	$\exp(\left[\log_{P}(Q),P\right])V\exp(-[\log_{P}(Q),P])$
$\operatorname{Exp}_{P}V$	$\left(\begin{array}[]{cc}PR&O\end{array}\right)\left(\begin{array}[]{c}\cos(\Sigma)\\ \sin(\Sigma)\end{array}\right)R^{\top}$ $V\stackrel{{\scriptstyle\mathrm{SVD}}}{{:=}}O\Sigma R^{\top}$	$\exp([V,P])P\exp(-[V,P])$
References	[ 25 , 5 ]	[ 3 , 5 ]

Appendix

Table 14 : Riemannian operators on the Grassmannian.

Method	Model	Mechanism	Formulation	Parameters	References
Möbius	$\mathbb{P}^{n}_{K}$	Tangent	$\operatorname{Exp}_{\mathbf{0}}(M\operatorname{Log}_{\mathbf{0}}(x))$	$M\in\mathbb{R}^{m\times n}$	[ 28 , Def. 3.2]
Klein	$\mathbb{K}^{n}_{K}$	Tangent	$\operatorname{Exp}_{\mathbf{0}}(M\operatorname{Log}_{\mathbf{0}}(x))$	$M\in\mathbb{R}^{m\times n}$	[ 51 , Thm. 9]
LFC	$\mathbb{H}^{n}_{K}$	Spacetime	$\displaystyle\left[\begin{array}[]{c}\frac{\sqrt{\\|Wx\\|^{2}-1/K}}{v^{\top}x}v^{\top}\\ W\end{array}\right]x$	$M\in\mathbb{R}^{m\times(n+1)}$ $v\in\mathbb{R}^{n+1}$	[ 14 , Sec 3.1]
NestFC	$\mathbb{H}^{n}_{K}$	Nested projection	$\displaystyle y=\frac{Wx}{\\|Wx\\|_{L}},\quad WJ_{n}W^{\top}=J_{m}$	$Q\in\mathrm{SO}(n)$ $\widetilde{P}\in\mathrm{St}(m,n)$ $\alpha\in\mathbb{R}$	[ 26 , Eq. (14) and Sec. 3.3]
Poincaré FC	$\mathbb{P}^{n}_{K}$	Poincaré	$w\left(1+\sqrt{1-K\\|w\\|^{2}}\right)^{-1}$ $w=\left((-K)^{-\frac{1}{2}}\sinh\left(\sqrt{-K}v_{k}(x)\right)\right)_{k=1}^{m}$ $v_{k}$ is defined by Shimizu et al. [66, Eq. (6)]	$\{z_{i}\in\mathbb{R}^{n}\}_{i=1}^{m}$ $\{\gamma_{i}\in\mathbb{R}\}_{i=1}^{m}$	[ 66 , Sec. 3.2]
Ours	$\mathbb{P}^{n}_{K},\mathbb{K}^{n}_{K},\mathbb{H}^{n}_{K}$	Riemannian	Thms. 4.1 and 4.2	$\{z_{i}\in\mathbb{R}^{n}\}_{i=1}^{m}$ $\{\gamma_{i}\in\mathbb{R}\}_{i=1}^{m}$	Thms. 4.1 and 4.2

Appendix

Table 15 : Comparison of hyperbolic linear layers. Here, we consider the transformation from an

n

-dimensional hyperbolic space to an

m

-dimensional one.

Method	Space	Complexity	Parameter dimension
Möbius [ 28 ]	$\mathbb{P}^{n}_{K}$	$O(Bnm)$	$nm$
Einstein [ 51 ]	$\mathbb{K}^{n}_{K}$	$O(Bnm)$	$nm$
LorentzTan	$\mathbb{H}^{n}_{K}$	$O(Bnm)$	$nm$
LFC [ 14 ]	$\mathbb{H}^{n}_{K}$	$O(Bnm)$	$m(n+1)+m+(n+1)+2$
NestFC [ 26 ]	$\mathbb{H}^{n}_{K}$	$\underline{O(n^{3}+Bn^{2})}$	$\underline{\frac{n(n-1)}{2}+mn-\frac{m(m+1)}{2}+1}$
Poincaré FC [ 66 ]	$\mathbb{P}^{n}_{K}$	$O(Bnm)$	$nm+m$

Appendix

Table 17 : Forward complexity and parameter dimensions of an

n

-to-

m

hyperbolic FC layer for a batch of

B

samples in the reduction setting

m\leq n

. The worst entry in each metric is underlined.

Step	Hyperparameter	Candidates
1	Feature normalization ( normalize_feats )	$0,1$
2	Activation	None, ReLU, LeakyReLU, ELU, Tanh
3	Learning rate	$0.001,0.005,0.008,0.01,0.02$
4	Gyro bias	$0,1$ for HFC-H only. NestFC skips this step
5	Gradient clipping	None, $0.5,1.0$
6	Attention and local aggregation	$(0,0),(1,1)$ for ( use_att , local_agg )

Appendix

Table 18 : Hyperparameter-search procedure.

Method	Disease	Airport	Pubmed	Cora
NHGCN-NestFC	78.28 ± 0.73	94.49 ± 0.04	94.70 ± 0.21	93.11 ± 0.40
NHGCN-HFC-H	97.37 ± 0.22	96.14 ± 0.27	96.18 ± 0.06	92.85 ± 0.24

Appendix

Table 19 : Link-prediction test AUC (%).

Method	Disease			Airport			Pubmed			Cora
Method	#Param	MiB	FitTime	#Param	MiB	FitTime	#Param	MiB	FitTime	#Param	MiB	FitTime
NHGCN-NestFC	738	81.10	27.039	776	120.29	43.292	257,952	3496.40	105.094	2,075,436	1266.51	94.458
NHGCN-HFC-H	450	92.88	19.854	465	134.34	44.638	7,785	3444.32	91.064	21,780	253.68	24.007

Appendix

Table 20 : Full-model trainable parameters, peak memory (MiB), and FitTime (ms/epoch) for NHGCN-NestFC and NHGCN-HFC-H.

Dataset	Model	$\theta$	Optimizer	Learning Rate
Radar	SPDNN-LEM	N/A	AMSGrad	$5e^{-3}$
	SPDNN-AIM	0.25	AMSGrad	$5e^{-4}$
	SPDNN-PEM	N/A	AMSGrad	$1e^{-2}$
	SPDNN-LCM	0.25	AMSGrad	$5e^{-4}$
	SPDNN-BWM	N/A	AMSGrad	$5e^{-4}$
HDM05	SPDNN-LEM	N/A	SGD	$5e^{-3}$

Appendix

Table 21 : Training hyperparameters in SPDNNs.

Method	FPHA
Method	Source	Ours	Gap
SPDNet (official code)	$88.79\pm 0.36$	$85.59\pm 0.72$	$-3.20$
SPDNetBN (official code)	$91.02\pm 0.25$	$89.33\pm 0.49$	$-1.69$
GyroLE	$94.61$	$90.73\pm 0.92$	$-3.88$
GyroLC	$82.43$	$76.10\pm 0.63$	$-6.33$
GyroAI	$93.39$	$89.60\pm 0.37$	$-3.79$

Appendix

Table 22 : FPHA accuracy (%): Source [ 55 , 56 ] , Ours, and Gap (Ours

-

Source; percentage points).

Method	Geometry	Radar	HDM05	FPHA	NTU60
SPDNet	N/A	0.66	0.50	0.28	3.08
SPDNetBN	AIM	1.25	0.94	0.58	6.14
SPDResNet-AIM	AIM	0.96	1.23	0.69	6.84
SPDResNet-LEM	LEM	0.77	0.55	0.30	3.17
SPDNetLieBN-AIM	AIM	1.21	1.15	0.97	8.85
SPDNetLieBN-LCM	LCM	1.10	1.11	0.59	5.96

Appendix

Table 23 : Training efficiency (seconds per epoch).

Input shape	#Conv	Kernels	Kernel sizes	Strides	Shape changes	Final SPD features
$[40,3,3]$	3	$[4,8,16]$	$[3,3,3]$	$[2,2,2]$	$[40,3,3]\to[19,4,3,3]\to[9,8,3,3]\to[4,16,3,3]$	$[64,3,3]$
$[20,6,6]$	3	$[4,8,16]$	$[2,2,2]$	$[2,2,1]$	$[20,6,6]\to[10,4,6,6]\to[5,8,6,6]\to[4,16,6,6]$	$[64,6,6]$
$[5,12,12]$	2	$[8,20]$	$[2,2]$	$[1,1]$	$[5,12,12]\to[4,8,12,12]\to[3,20,12,12]$	$[60,12,12]$

Appendix

Table 24 : SPD CNN architectures with local receptive fields and shared kernels.

Method	$[40,3,3]$		$[20,6,6]$		$[5,12,12]$
Method	Accuracy	FitTime	Accuracy	FitTime	Accuracy	FitTime
ManifoldNet	77.89 ± 1.56	13.12	73.27 ± 0.68	11.58	80.75 ± 1.23	11.50
MVC-Net	81.04 ± 1.72	7.88	78.05 ± 0.23	8.18	81.15 ± 0.34	7.87
SPDConvNet-LEM	97.68 ± 0.32	1.58	95.86 ± 0.95	1.55	96.91 ± 0.18	$1.03$
SPDConvNet-LCM	96.51 ± 0.66	$1.54$	94.40 ± 0.50	$1.22$	97.20 ± 0.65	1.25
SPDConvNet-PEM	97.44 ± 0.56	1.82	94.45 ± 0.78	1.53	96.16 ± 0.91	2.34

Appendix

Table 25 : Radar classification accuracy (%) and FitTime (s/epoch) under different architectures.

Learning rate	$5\times 10^{-5}$	$10^{-4}$	$2\times 10^{-4}$	$5\times 10^{-4}$	$10^{-3}$	$2\times 10^{-3}$	$5\times 10^{-3}$	$10^{-2}$
Accuracy (%)	55.20 ± 2.58	59.73 ± 1.31	63.52 ± 1.44	69.55 ± 2.46	73.28 ± 1.42	76.08 ± 1.08	77.89 ± 1.56	77.12 ± 1.30

Appendix

Table 26 : ManifoldNet learning-rate ablation under the

[40,3,3]

input setting.

Explore similar work

Jul 21, 2026cs.LG

Riemannian Deep Learning: Modules, Networks, and Geometries

Deep neural networks on manifold-valued representations have attracted growing interest, but many basic components remain tied to specific manifolds, rely on Euclidean approximations, or require costly and numerically fragile geometric operations. This thesis develops a unified framework for Riemannian deep learning from three complementary perspectives: reusable neural modules, manifold-specific network architectures, and the design of underlying geometries. It generalizes batch normalization from Euclidean spaces and individual manifolds to broad classes of Lie groups and gyrogroups, and extends multinomial logistic regression from Euclidean space to SPD manifolds and then to general Riemannian manifolds. It further develops neural networks for several important geometric representations, including an unconstrained model of hyperbolic space, Busemann-based hyperbolic learning, and full-rank correlation matrices. Finally, it introduces adaptive and computationally efficient Riemannian metrics on SPD manifolds, including learnable Log-Euclidean geometries and fast, stable Cholesky-based geometries. The proposed methods are supported by theoretical analysis and validated through numerical experiments and applications in vision, signal processing, graph learning, and genomics.

Ziheng Chen

Doctoral School in Information and Communication Technology, Università di Trento

Data Manifold Geometric Deep Learning Hyperbolic Learning

Ask PDF

May 1, 2026cs.LG

Batch Normalization for Neural Networks on Complex Domains

Riemannian neural networks have proven effective in solving a variety of machine learning tasks. The key to their success lies in the development of principled Riemannian analogs of fundamental building blocks in deep neural networks (DNNs). Among those, Riemannian batch normalization (BN) layers have shown to enhance training stability and improve accuracy. In this paper, we propose BN layers for neural networks on complex domains. The proposed layers have close connections with existing Riemannian BN layers. We derive essential components for practical implementations of BN layers on some complex domains which are less studied in previous works, e.g., the Siegel disk domain. We conduct experiments on radar clutter classification, node classification, and action recognition demonstrating the efficacy of our method.

Xuan Son Nguyen, Nistor Grozavu

ETIS, UMR 8051, CY Cergy Paris University, ENSEA, CNRS, Cergy, France

Batch Normalization Neural Network

Ask PDF

May 18, 2026cs.LG

Riemannian Networks over Full-Rank Correlation Matrices

Representations on the Symmetric Positive Definite (SPD) manifold have garnered significant attention across different applications. In contrast, the manifold of full-rank correlation matrices, a normalized alternative to SPD matrices, remains largely underexplored. This paper introduces Riemannian networks over the correlation manifold, leveraging five recently developed correlation geometries. We systematically extend basic layers, including Multinomial Logistic Regression (MLR), Fully Connected (FC), and convolutional layers, to these geometries. Besides, we present methods for accurate backpropagation for two correlation geometries. Experiments comparing our approach against existing SPD and Grassmannian networks demonstrate its effectiveness.

Ziheng Chen, Xiaojun Wu, Bernhard Schölkopf +1

Department of Information Engineering and Computer Science, University of Trento, Trento, Italy · Max Planck Institute for Intelligent Systems, Tübingen, Germany · School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China

Full-Rank Correlation Matrices High-Order Correlations Rectified Linear Unit Networks Backpropagation

Ask PDF