Large Scale GNN

Large-scale Graph Neural Networks (GNNs) aim to overcome the computational and memory limitations of training GNNs on massive graphs, crucial for applications like recommendation systems and drug discovery. Current research focuses on optimizing data transfer and partitioning strategies across multiple GPUs and nodes, employing techniques like cached operator reordering and in-storage processing to improve efficiency. These advancements enable the training of significantly larger GNN models, leading to more accurate and powerful models for various real-world applications that previously were computationally infeasible.

Papers

July 21, 2024

LSM-GNN: Large-scale Storage-based Multi-GPU GNN Training by Optimizing Data Transfer Scheme
Jeongmin Brian Park, Kun Wu, Vikram Sharma Mailthody, Zaid Quresh, Scott Mahlke, Wen-mei Hwu
Graph Neural Network Dfa GNN Data Prefetching Large Scale GNN Data Transfer

August 29, 2023

An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Training
Nikolai Merkel, Daniel Stoll, Ruben Mayer, Hans-Arno Jacobsen
Graph Neural Network Large Scale Graph Graph Partitioning Experimental Comparison GNN Performance Large Scale GNN

August 23, 2023

Cached Operator Reordering: A Unified View for Fast GNN Training
Julia Bazinska, Andrei Ivanov, Tal Ben-Nun, Nikoli Dryden, Maciej Besta, Siyuan Shen, Torsten Hoefler
Graph Neural Network Graph Convolutional Network Unified View GNN Training GNN Layer GNN Computation Large Scale GNN

June 28, 2023

Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses
Jeongmin Brian Park, Vikram Sharma Mailthody, Zaid Qureshi, Wen-mei Hwu
Graph Neural Network Open Sampling GNN Framework Training Throughput Training Pipeline Consistent Aggregation Large Scale GNN

May 23, 2023

The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
Jana Vatter, Ruben Mayer, Hans-Arno Jacobsen
Deep Learning Graph Neural Network Specie Evolution GNN Based Service Graph Processing Large Scale GNN Scalable GNN

July 22, 2022

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules
Jong Youl Choi, Pei Zhang, Kshitij Mehta, Andrew Blanchard, Massimiliano Lupo Pasini
Convolutional Neural Network Graph Drawing Gene Level GNN Molecular Graph Known Molecule Graph Convolutional Neural Network Large Scale Graph Datasets Scalable Training Large Scale GNN

June 14, 2022

GraphFM: Improving Large-Scale GNN Training via Feature Momentum
Haiyang Yu, Limei Wang, Bokun Wang, Meng Liu, Tianbao Yang, Shuiwang Ji
Node Classification Node Representation Large Scale Graph Datasets Large Scale GNN

May 10, 2022

SmartSAGE: Training Large-scale Graph Neural Networks using In-Storage Processing Architectures
Yunjae Lee, Jinha Chung, Minsoo Rhu
Memory Efficient Large Scale GNN

Large Scale GNN

Papers

LSM-GNN: Large-scale Storage-based Multi-GPU GNN Training by Optimizing Data Transfer Scheme

An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Training

Cached Operator Reordering: A Unified View for Fast GNN Training

Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses

The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

GraphFM: Improving Large-Scale GNN Training via Feature Momentum

SmartSAGE: Training Large-scale Graph Neural Networks using In-Storage Processing Architectures