Batch Before You Lift: Scalable Topological Deep Learning on Large Graphs

David Leko; Luka Benić; Guillermo Bernárdez; Nina Miolane; Olga Fink; Lev Telyatnikov

cs.LGOct 8, 2026

Batch Before You Lift: Scalable Topological Deep Learning on Large Graphs

Authors: David Leko, Luka Benić, Guillermo Bernárdez, Nina Miolane, Olga Fink, Lev Telyatnikov

Organizations: IMOS Lab, EPFL, Lausanne, Switzerland · University of Zagreb, Zagreb, Croatia · UC Santa Barbara, Santa Barbara, USA · Capital One

Abstract

Topological Deep Learning extends graph-based learning to higher-order domains, such as hypergraphs, cellular, and simplicial complexes. These domains are typically constructed from patterns in an input graph through a process of graph lifting. Full-domain training constructs and stores the complete lifted representation before model execution. On large and dense datasets like Reddit (233k nodes and 57.3M edges), this global materialization becomes a severe computational bottleneck, often rendering training infeasible. To address this limitation, we introduce Cluster-TNN, a domain-agnostic framework that avoids this bottleneck by lifting locally instead. After partitioning the input graph during preprocessing, at runtime Cluster-TNN dynamically samples groups of node clusters, reconstructs their induced subgraphs to form mini-batches, and applies the chosen lifting within each mini-batch. Retaining all edges among sampled nodes preserves the connectivity needed to construct higher-order structures across clusters, producing topological mini-batches that existing Topological Neural Networks can process directly. Across 21 matched comparisons with full-graph execution, Cluster-TNN reduces peak GPU memory in every configuration, by 83.2% on average while maintaining competitive predictive performance. Notably, such a reduction enables, to our knowledge, the first training of multiple different higher-order Topological Neural Networks on large datasets such as Reddit and OGBN Products. These results establish Cluster-TNN as a general strategy for scaling Topological Deep Learning beyond the limitations of global domain construction.

Figures & tables

Figure 1: Comparison of Cluster-TNN training (ours) and standard full-graph TDL training. Both workflows start from the same featured undirected graph. a Cluster-TNN partitions the graph once per run and at each training step samples up to

q

clusters, reconstructs their induced subgraph, and performs lifting and model execution only on the resulting mini-batch. b Standard full-graph training instead constructs and processes the complete lifted domain throughout training. c Trained models support both mini-batch and full-graph inference.

Figure 2: Structural support recovery as a function of

q

. Cora Full uses

K=32

and Coauthor Physics

K=2\,000

. Curves show means over ten independent reshuffling runs. See Appendix C.13.1 for more details.

Figure 3: Cluster-TNN reduces peak GPU memory across all model-dataset pairs. Each circle represents one model, positioned by the mean ratios of Cluster-TNN to full-graph peak GPU memory and total runtime across five seeds. Moving left indicates memory savings, whereas moving down indicates shorter runtime. The lower-left region marks improvements in both measures. Marker size represents the absolute test-performance difference in percentage points, and color distinguishes performance gains from losses. Dashed lines mark parity with full-graph execution.

Figure 4: GPU memory scaling on Cora Full with 512 hidden channels for

K=64

and

q=8

. Peak GPU memory is reported as mean

\pm

standard deviation across five seeds. Network depth counts TopoTune blocks and SCCNN layers.

	Cora Full		Amazon Ratings		Questions		Coauthor Physics	Reddit	OGBN Products
Model	Full-graph	Cluster-TNN	Full-graph	Cluster-TNN	Full-graph	Cluster-TNN	Cluster-TNN	Cluster-TNN	Cluster-TNN
GCN	$70.18\pm 0.42$	$69.19\pm 0.66$	$48.29\pm 1.31$	$49.62\pm 0.96$	$76.69\pm 1.18$	$73.83\pm 2.01$	$\mathbf{96.68\pm 0.08}$	$\mathbf{92.62\pm 0.10}$	$85.81\pm 0.04$
EDHNN	$69.74\pm 0.89$	$67.06\pm 1.19$	$48.41\pm 1.18$	$\mathbf{51.41\pm 0.72}$	$75.73\pm 0.92$	$\mathbf{76.23\pm 1.05}$	$96.31\pm 0.32$	$92.17\pm 0.73$	$86.51\pm 0.25$
UniGNN	$68.44\pm 0.79$	$67.61\pm 0.56$	$47.05\pm 1.76$	$47.24\pm 0.70$	$69.44\pm 1.68$	$71.09\pm 0.98$	$94.97\pm 0.37$	$91.28\pm 0.16$	$85.87\pm 0.07$
CWN	$60.50\pm 0.60$	$62.66\pm 1.11$	$44.07\pm 0.64$	$46.25\pm 0.67$	$68.21\pm 3.48$	$70.20\pm 0.88$	$95.80\pm 0.24$	$91.29\pm 0.18$	$83.93\pm 0.06$
TopoTune	$46.40\pm 0.47$	$56.45\pm 1.42$	$43.18\pm 1.03$	$43.27\pm 0.68$	$71.53\pm 1.73$	$73.61\pm 1.88$	$95.22\pm 0.20$	$89.55\pm 0.09$	$83.79\pm 0.55$

Table 1: Test performance under full-graph and Cluster-TNN settings. We report AUROC for Questions and accuracy for all other datasets. Cluster-TNN uses ten-recombination ensemble mini-batch inference, whereas full-graph runs use full-graph inference. Values are mean

\pm

sample standard deviation across five seeds. Boldface indicates best-performing model for each setting.

Cora Full

Amazon Ratings

Questions

Model

Partitioner

Edges

Ensemble

Full-graph

Partitioner

Edges

Ensemble

Full-graph

Partitioner

Edges

Ensemble

Full-graph

GCN

+0.98

+0.08

+2.51

+3.44

+2.24

+0.25

+0.05

+0.08

+0.52

+0.99

+0.65

+2.71

EDHNN

+6.28

+1.72

+1.55

+2.59

+2.88

-0.44

+0.45

+0.33

+1.25

+1.04

+0.42

+0.22

UniGNN

+6.45

+0.63

+0.04

+0.30

+2.47

-0.43

+0.33

+0.34

-1.01

+1.10

+1.35

+0.99

CWN

+1.91

+0.66

+2.57

+5.37

+0.18

+0.23

+0.10

+0.08

-0.09

+0.56

+0.14

+0.15

TopoTune

-3.86

+10.01

+4.65

+4.75

-2.40

+0.41

+1.56

+0.48

+1.15

+1.49

+0.30

-0.58

Table 2: Partitioning and inference ablations (shaded columns). Partitioner compares METIS with random partitioning, and Edges compares preserving with removing cross-cluster edges. Ensemble and Full-graph compare the respective inference protocols with single-pass mini-batch inference. Entries report mean paired differences in test performance (percentage points) over five seeds.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Input graph			Cellular 2-cells		Simplicial 2-cells
Dataset	Nodes	Edges	Mean node degree	Count	Per node	Count	Per node
Questions	$48\,921$	$153\,540$	$6.28$	$24\,405$	$0.50$	$110\,209$	$2.25$
Amazon Ratings	$24\,492$	$93\,050$	$7.60$	$63\,667$	$2.60$	$110\,765$	$4.52$
Cora Full	$19\,793$	$63\,421$	$6.41$	$24\,306$	$1.23$	$48\,386$	$2.44$
Coauthor Physics	$34\,493$	$247\,962$	$14.38$	$102\,152$	$2.96$	$468\,550$	$13.58$
Reddit	$232\,965$	$57\,307\,946$	$491.99$	$40\,582\,059$	$174.20$	$8\,360\,338\,411$	$35\,886.67$

Appendix

Table 3: Structural characteristics of the input graphs and their lifted domains. The edge column counts each undirected relation once, and mean node degree is derived as

2|\mathcal{E}|/|\mathcal{V}|

. Per-node values are the corresponding 2-cell counts divided by the number of input nodes. Cellular 2-cells are retained basis cycles of length at most nine, whereas simplicial 2-cells are filled graph triangles.

Dataset	Input width $d_{0}$	$k=1$ hypergraph	Cycle cell complex	Clique simplicial complex
Questions	$301$	$0.118$ GB	$0.273$ GB	$0.376$ GB
Amazon Ratings	$300$	$0.059$ GB	$0.217$ GB	$0.274$ GB
Cora Full	$8\,710$	$1.379$ GB	$3.746$ GB	$4.585$ GB
Coauthor Physics	$8\,415$	$2.322$ GB	$12.946$ GB	$25.279$ GB
Reddit	$602$	$1.122$ GB	$236.280$ GB	$20.270$ TB
OGBN Products	$100$	$1.959$ GB	$39.414$ GB	$274.156$ GB

Appendix

Table 4: Estimated memory required to store the float32 input feature matrices of each full-domain lifting. Values use the input feature dimension of each dataset. Observed preprocessing memory is reported in Table 5 .

Dataset	Lifting	Model	Additional operators	Preprocessing time (s)	Peak CPU memory (GB)
Amazon Ratings	$k=1$ neighbourhood hypergraph	EDHNN	None beyond the node-hyperedge incidence	$0.42\pm 0.07$	$1.00\pm 0.01$
Amazon Ratings	$k=1$ neighbourhood hypergraph	UniGNN	None beyond the node-hyperedge incidence	$0.39\pm 0.02$	$1.01\pm 0.01$
Amazon Ratings	Cycle-basis cell	CWN	Rank-1 upper adjacency	$5.7\pm 0.2$	$1.38\pm 0.00$
Amazon Ratings	Cycle-basis cell	TopoTune	Rank-1 and two-hop rank-0 upper adjacencies	$8.5\pm 0.2$	$1.39\pm 0.02$
Amazon Ratings	Dimension-2 clique complex	SCN	None beyond the rank-1 and rank-2 incidences	$4.3\pm 0.3$	$1.44\pm 0.00$
Amazon Ratings	Dimension-2 clique complex	SCCNN	None beyond the rank-1 and rank-2 incidences	$4.3\pm 0.3$	$1.44\pm 0.00$

Appendix

Table 5: Available full-graph preprocessing time and peak CPU memory measurements for the evaluated dataset-lifting-model configurations. Models are distinguished because they may request different operators from the same lifted domain. Values are reported as the mean

\pm

standard deviation across five runs. Memory usage is reported in decimal GB.

Figure 5: Cluster-TNN consistently reduces peak GPU memory during training and evaluation, whereas preprocessing memory varies by configuration and can increase when partition construction exceeds the cost of full-graph lifting. Values are means across five seeds.

Model	Setting	$\boldsymbol{K}$	$\boldsymbol{q}$	Preprocessing	Train	Final evaluation	Total
GCN	Full-graph	-	-	$4.0\pm 2.5$	$222.3\pm 57.4$	$3.4\pm 0.2$	$229.7\pm 57.5$
	Cluster-TNN	$32$	$8$	$11.2\pm 1.1$	$83.7\pm 8.8$	$47.2\pm 0.5$	$142.1\pm 8.8$
EDHNN	Full-graph	-	-	$8.0\pm 0.8$	$438.7\pm 114.3$	$7.2\pm 0.3$	$453.9\pm 113.8$
	Cluster-TNN	$32$	$4$	$11.2\pm 1.1$	$180.5\pm 40.3$	$48.7\pm 0.9$	$240.3\pm 40.4$
UniGNN	Full-graph	-	-	$8.4\pm 0.6$	$343.8\pm 71.9$	$7.1\pm 0.1$	$359.3\pm 71.8$
	Cluster-TNN	$32$	$4$	$11.2\pm 1.1$	$150.7\pm 21.9$	$50.4\pm 1.0$	$212.2\pm 22.3$

Appendix

Table 6: End-to-end runtime in seconds under full-graph and Cluster-TNN settings. Preprocessing, training, and final evaluation are reported separately. Training is the complete fit call, including periodic validation. Final evaluation spans the end of fitting through checkpoint validation and test inference. Total is the sum of these non-overlapping components. For Cluster-TNN inference, we use the ensemble protocol and average predictions over ten cluster recombinations. Values are mean

\pm

sample standard deviation in seconds across five seeds.

Subset	Count	Full-graph sum (s)	Cluster-TNN sum (s)	Pooled change
Shorter partitioned runtime	11	$2\,582.9$	$1\,174.0$	$-54.5\%$
Longer partitioned runtime	10	$3\,371.1$	$7\,934.1$	$+135.4\%$
All matched configurations	21	$5\,954.0$	$9\,108.1$	$+53.0\%$

Appendix

Table 7: Pooled total-runtime changes calculated from the configuration-level means in Table 6 . Each sum combines the five-seed mean total runtime of the configurations in the indicated subset. Subsets are defined by the direction of the total-runtime change. Partitioned-only configurations are excluded.

				Full-graph		Cluster-TNN
Dataset	Model	$\boldsymbol{K}$	$\boldsymbol{q}$	Peak CPU memory (GB)	Peak GPU memory (GB)	Peak CPU memory (GB)	Peak GPU memory (GB)
	GCN	$32$	$8$	$2.96\pm 0.01$	$6.95\pm 0.00$	$3.64\pm 0.03$	$1.80\pm 0.01$
	EDHNN	$32$	$4$	$3.68\pm 0.02$	$7.65\pm 0.00$	$3.64\pm 0.03$	$1.01\pm 0.00$
	UniGNN	$32$	$4$	$3.71\pm 0.01$	$7.65\pm 0.00$	$3.64\pm 0.03$	$1.02\pm 0.00$
Cora Full	CWN	$32$	$4$	$7.63\pm 0.01$	$24.14\pm 0.00$	$3.89\pm 0.07$	$3.65\pm 0.09$
	TopoTune	$64$	$8$	$7.61\pm 0.03$	$24.21\pm 0.00$	$3.98\pm 0.12$	$3.53\pm 0.09$

Appendix

Table 8: Run-wide peak CPU and GPU memory under full-graph and Cluster-TNN execution. Values are mean

\pm

sample standard deviation in decimal GB across five seeds. CPU values are main-process RSS peaks. GPU values are CUDA allocated-memory peaks.

Dataset	Model	$\boldsymbol{K}$	$\boldsymbol{q}$	Peak CPU memory (GB)	Peak GPU memory (GB)
	GCN	$500$	$20$	$6.68\pm 0.03$	$0.51\pm 0.00$
	EDHNN	$2\,000$	$30$	$6.07\pm 0.05$	$0.24\pm 0.00$
	UniGNN	$2\,000$	$20$	$6.07\pm 0.05$	$0.16\pm 0.00$
Coauthor Physics	CWN	$2\,000$	$20$	$6.07\pm 0.05$	$0.38\pm 0.01$
	TopoTune	$2\,000$	$20$	$6.07\pm 0.05$	$0.38\pm 0.01$
	SCN	$2\,000$	$20$	$6.07\pm 0.05$	$0.67\pm 0.05$

Appendix

Table 9: Cluster-TNN run-wide peak memory on Coauthor Physics, Reddit, and OGBN Products, using the measurement definitions of Table 8 . CPU values are main-process RSS peaks. GPU values are CUDA allocated-memory peaks. Both are reported in decimal GB.

				Full-graph				Cluster-TNN
Dataset	Model	$\boldsymbol{K}$	$\boldsymbol{q}$	$\boldsymbol{E^{*}}$	$\boldsymbol{U^{*}}$	Val.	Test	$\boldsymbol{E^{*}}$	$\boldsymbol{U^{*}}$	Val.	Test
Cora Full	GCN	32	8	$94\pm 30$	$94\pm 30$	$70.02\pm 1.23$	$70.18\pm 0.42$	$44\pm 10$	$176\pm 39$	$68.42\pm 1.63$	$69.19\pm 0.66$
	EDHNN	$32$	$4$	$82\pm 29$	$82\pm 29$	$70.07\pm 0.97$	$69.74\pm 0.89$	$59\pm 19$	$472\pm 153$	$66.89\pm 1.15$	$67.06\pm 1.19$
	UniGNN	$32$	$4$	$58\pm 18$	$58\pm 18$	$68.92\pm 1.32$	$68.44\pm 0.79$	$44\pm 10$	$352\pm 82$	$67.43\pm 1.23$	$67.61\pm 0.56$
	CWN	$32$	$4$	$83\pm 12$	$83\pm 12$	$61.29\pm 0.82$	$60.50\pm 0.60$	$57\pm 3$	$456\pm 22$	$62.37\pm 1.04$	$62.66\pm 1.11$
	TopoTune	$64$	$8$	$22\pm 6$	$22\pm 6$	$46.87\pm 0.62$	$46.40\pm 0.47$	$67\pm 21$	$536\pm 171$	$55.92\pm 0.92$	$56.45\pm 1.42$

Appendix

Table 10: Statistical-efficiency for full-graph and Cluster-TNN training.

E^{*}

is the epoch selected by the validation metric, and

U^{*}

is the number of optimizer updates completed at that checkpoint. Validation and test results are reported. The metric is accuracy for Cora Full and Amazon Ratings and AUROC for Questions. Full-graph uses one full-domain test pass, whereas Cluster-TNN uses the ten-pass ensemble protocol. Each entry is the mean

\pm

sample standard deviation across five seeds.

Dataset	Model	$\boldsymbol{K}$	$\boldsymbol{q}$	$\boldsymbol{E^{*}}$	$\boldsymbol{U^{*}}$	Val.	Test
	GCN	$500$	$20$	$39\pm 27$	$975\pm 681$	$96.32\pm 0.12$	$96.68\pm 0.08$
	EDHNN	$2\,000$	$30$	$58\pm 20$	$3886\pm 1332$	$96.13\pm 0.11$	$96.31\pm 0.32$
	UniGNN	$2\,000$	$20$	$68\pm 17$	$6800\pm 1681$	$94.66\pm 0.19$	$94.97\pm 0.37$
Coauthor Physics	CWN	$2\,000$	$20$	$34\pm 24$	$3400\pm 2434$	$95.69\pm 0.10$	$95.80\pm 0.24$
	TopoTune	$2\,000$	$20$	$18\pm 7$	$1800\pm 671$	$94.70\pm 0.27$	$95.22\pm 0.20$
	SCN	$2\,000$	$20$	$32\pm 20$	$3200\pm 1956$	$95.84\pm 0.20$	$96.22\pm 0.22$

Appendix

Table 11: Cluster-TNN statistical efficiency on Coauthor Physics, Reddit, and OGBN Products, using the checkpoint and inference definitions of Table 10 .

E^{*}

is the selected epoch,

U^{*}

the completed optimizer updates, and validation and test scores are accuracies (%).

Figure 6: Fraction of full-graph reference structures whose supporting nodes appear together in at least one mini-batch within 200 epochs, as a function of the number of clusters per mini-batch

q

. Panels show Cora Full and Amazon Ratings (

K=32

, top row), Questions (

K=500

, bottom left), and Coauthor Physics (

K=2\,000

, bottom right). Each dataset uses one fixed METIS partition. Curves show means over ten independent reshuffling runs. Shading indicates

\pm 1

standard deviation, generally narrower than the plotted lines. Cellular recovery refers to the supporting nodes of fixed full-graph cycle-basis cells, not their selection by a recomputed local basis. Vertical scales begin at

50\%

for Cora Full,

75\%

for Amazon Ratings, and

0\%

for the other datasets.

Figure 7: Comparison of empirical and analytical structural recovery on Cora Full with

K=32

and

q\in\{2,4,8,16\}

. Empirical coverage is shown as the mean

\pm 1

sample standard deviation across ten independent runs and compared with the analytical coverage from Eq. ( 43 ) over 200 epochs.

Figure 8: Analytical entropy of cumulative support availability on Cora Full (

K=32

). a Eq. ( 44 ) for

T\leq 10^{4}

. b Blue circles mark peak epochs, while black squares mark the final crossing below

1\%

of peak entropy.

Figure 9: Peak GPU memory under Cluster-TNN and full-graph execution on three datasets. Rows correspond to models, while columns vary the number of partitions $K$ and clusters per mini-batch $q$ , with percentages indicating $q/K$ . The rightmost column shows the full-graph baseline. Each dataset has a separate color scale.

Dataset	Metric	Input feature dimension	Classes
Cora Full	Accuracy	$8\,710$	$70$
Amazon Ratings	Accuracy	$300$	$5$
Questions	AUROC	$301$	$2$
Coauthor Physics	Accuracy	$8\,415$	$5$
Reddit	Accuracy	$602$	$41$
OGBN Products	Accuracy	$100$	$47$

Appendix

Table 12: Dataset tasks, input feature dimensions, and class counts. Structural counts are reported separately in Table 3 .

Dataset	Candidate $K$ values	Candidate $q$ values
Questions	$\{500,1\,000,1\,500,2\,000\}$	$\{20,30,40,50\}$
Amazon Ratings	$\{32,64,128,256\}$	$\{4,8,16,32\}$
Cora Full	$\{32,64,128,256\}$	$\{4,8,16,32\}$
Coauthor Physics	$\{500,1\,000,1\,500,2\,000\}$	$\{20,30,40,50\}$
Reddit	$\{7\,000,10\,000\}$	$\{20,25\}$
OGBN Products	$\{15\,000,20\,000\}$	$\{20,25\}$

Appendix

Table 13: Dataset-specific partition search spaces.

K

denotes the total number of graph partitions, and

q

denotes the number of clusters combined into one training mini-batch. These parameters are optimized jointly with the model hyperparameters.

Dataset	Protocol	Learning Rate	Weight Decay	Hidden Channels	Projection Dropout	$\mathbf{K}$	$\mathbf{q}$	Dropout
GCN
Questions	Full-graph	$10^{-2}$	$10^{-4}$	$128$	$0.3$			$0.5$
	Cluster-TNN	$10^{-4}$	$0$	$128$	$0.2$	$500$	$20$	$0.2$
Amazon Ratings	Full-graph	$10^{-2}$	$10^{-3}$	$128$	$0$			$0$
	Cluster-TNN	$10^{-2}$	$10^{-4}$	$128$	$0.3$	$64$	$4$	$0$
Cora Full	Full-graph	$10^{-3}$	$10^{-5}$	$128$	$0.3$			$0$

Appendix

Table 14: Selected hyperparameters for full-graph and Cluster-TNN training. Shading denotes parameters that do not apply to the corresponding model or protocol.

Explore similar work

Differentiable Lifting for Topological Neural Networks
Aug 2, 2026Jorge Luiz Franco, Gabriel Duarte, Alexander Nikitin +3Graph Structure Learning Graph Representation Learning
University of São Paulo, Instituto Curvelo · Federal Institute of Ceará · Aalto University +3
Scaling Higher-Order Graph Learning with Maximal Clique Complexes
May 29, 2026Antoine Vialle, Aref Einizade, Fragkiskos D. Malliaros +1Graph Neural Networks Graph Representation Learning
LTCI, Télécom Paris · Institut Polytechnique de Paris · Palaiseau, France +4
TopoFormer: Topology Meets Attention for Graph Learning
Jul 30, 2026Md Joshem Uddin, Astrit Tola, Cuneyt Gurcan Akcora +1Graph Representation Learning Graph Classification
Department of Mathematical Science The University of Texas at Dallas Richardson, TX 75080, USA · Department of Mathematics Florida State University Tallahassee, FL 32306, USA · AI Initiative University of Central Florida Orlando, FL 32816, USA

Aug 2, 2026, cs.LGJ/K move · Enter open · S save

Differentiable Lifting for Topological Neural Networks

Jorge Luiz Franco, Gabriel Duarte, Alexander Nikitin, Moacir Ponti+2

University of São Paulo, Instituto Curvelo · Federal Institute of Ceará · Aalto University+3

Graph Structure Learning Graph Representation Learning Topological Representation Learning

Open paper AskPDF arXiv

Topological neural networks (TNNs) enable leveraging high-order structures on graphs (e.g., cycles and cliques) to boost the expressive power of message-passing neural networks. In turn, however, these structures are typically identified a priori through an unsupervised graph lifting operation. Notwithstanding, this choice is crucial and may have a drastic impact on a TNN's performance on downstream tasks. To circumvent this issue, we propose

\partial

lift (DiffLift), a general framework for learning graph liftings to hypergraphs and cellular- and simplicial complexes in an end-to-end fashion. In particular, our approach leverages learned vertex-level latent representations to identify and parameterize distributions over candidate higher-order cells for inclusion. This results in a scalable model which can be readily integrated into any TNN. Our experiments show that

\partial

lift outperforms existing lifting methods on multiple benchmarks for graph and node classification across different TNN architectures. Notably, our approach leads to gains of up to 45% over static liftings, including both connectivity- and feature-based ones.

Abstract

Explore similar work

Differentiable Lifting for Topological Neural Networks

Scaling Higher-Order Graph Learning with Maximal Clique Complexes

TopoFormer: Topology Meets Attention for Graph Learning