GNN Training

Training Graph Neural Networks (GNNs) efficiently on massive graphs is a critical challenge driving current research. Efforts focus on optimizing various stages of the training process, including subgraph sampling strategies, efficient data I/O and memory management (especially leveraging CPU-GPU heterogeneous environments and persistent memory), and communication reduction techniques for distributed training. These advancements aim to improve both the speed and scalability of GNN training, enabling the application of these powerful models to increasingly larger and more complex datasets in diverse fields like recommendation systems and knowledge graph reasoning.

Papers

November 6, 2024

Efficient Message Passing Architecture for GCN Training on HBM-based FPGAs with Orthogonal Topology On-Chip Networks
Qizhe Wu, Letian Zhao, Yuchen Gui, Huawen Liang Xiaotian Wang
Field Programmable Gate Array Message Passing GNN Training Network Bottleneck Network on Chip

November 2, 2024

Using Half-Precision for GNN Training
Arnab Kanti Tarafder, Yidong Gong, Pradeep Kumar
Gene Level GNN GNN Model Low Precision GNN Training GNN Based Service

October 9, 2024

TCGU: Data-centric Graph Unlearning based on Transferable Condensation
Fan Li, Xiaoyang Wang, Dawei Cheng, Wenjie Zhang, Ying Zhang, Xuemin Lin
GNN Model GNN Training Approximate Unlearning Graph Unlearning

October 7, 2024

Haste Makes Waste: A Simple Approach for Scaling Graph Neural Networks
Rui Xue, Tong Zhao, Neil Shah, Xiaorui Liu
Graph Neural Network Graph Representation Learning Large Scale Graph GNN Training Simple Approach Efficient Waste

September 23, 2024

FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
Zeyu Zhu, Peisong Wang, Qinghao Hu, Gang Li, Xiaoyao Liang, Jian Cheng
Graph Neural Network Large Scale Large Graph GNN Training GNN Based Service GPU Implementation GPU Based Parallel

August 23, 2024

HGNAS: Hardware-Aware Graph Neural Architecture Search for Edge Devices
Ao Zhou, Jianlei Yang, Yingjie Qi, Tong Qiao, Yumeng Shi, Cenlin Duan, Weisheng Zhao, Chunming Hu
Edge Device Memory Efficient GNN Training GNN Framework Hardware Aware Graph Neural Architecture Search

August 21, 2024

Slicing Input Features to Accelerate Deep Learning: A Case Study with Graph Neural Networks
Zhengjia Xu, Dingyang Lyu, Jinghui Zhang
Deep Learning Graph Neural Network Case Study Large Scale Graph GNN Training Input Feature Full Batch Gradient Descent

June 25, 2024

Distributed Training of Large Graph Neural Networks with Variable Communication Rates
Juan Cervino, Md Asadullah Turja, Hesham Mostafa, Nageen Himayat, Alejandro Ribeiro
Graph Neural Network Training Data Gene Level GNN Large Graph GNN Training

June 20, 2024

Reducing Memory Contention and I/O Congestion for Disk-based GNN Training
Qisheng Jiang, Lei Jia, Chundong Wang
GNN Training Grid Congestion GNN Based Service Resource Contention

May 10, 2024

Disttack: Graph Adversarial Attacks Toward Distributed GNN Training
Yuxiang Zhang, Xin Liu, Meng Wu, Wei Yan, Mingyu Yan, Xiaochun Ye, Dongrui Fan
Graph Neural Network Adversarial Attack Gene Level GNN GNN Training Graph Adversarial Attack

May 8, 2024

DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Training
Renjie Liu, Yichuan Wang, Xiao Yan, Zhenkun Cai, Minjie Wang, Haitian Jiang, Bo Tang, Jinyang Li
Graph Neural Network Large Graph GNN Training Model Accuracy

April 2, 2024

CATGNN: Cost-Efficient and Scalable Distributed Training for Graph Neural Networks
Xin Huang, Weipeng Zhuo, Minh Phu Vuong, Shiju Li, Jongryool Kim, Bradley Rees, Chul-Ho Lee
Graph Neural Network Training Data Potential Scalability GNN Architecture Large Graph GNN Training Cost Effective

March 9, 2024

Task-Oriented GNNs Training on Large Knowledge Graphs for Accurate and Efficient Modeling
Hussein Abdallah, Waleed Afandi, Panos Kalnis, Essam Mansour
Knowledge Graph Heterogeneous Graph Task Oriented Heterogeneous Graph Neural Network GNN Training Efficient Model Large Knowledge Graph Task Graph

November 22, 2023

NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments
Xin Ai, Qiange Wang, Chunyu Cao, Yanfeng Zhang, Chaoyi Chen, Hao Yuan, Yu Gu, Ge Yu
State of the Art GNN Model GNN Training GNN Based Service Heterogeneous GPU GPU Training

October 31, 2023

Privacy-preserving design of graph neural networks with applications to vertical federated learning
Ruofan Wu, Mingyang Zhang, Lingjuan Lyu, Xiaolong Xu, Xiuquan Hao, Xinyi Fu, Tengfei Liu, Tianyi Zhang, Weiqiang Wang
Neural Network Financial Application Graph Drawing Product Design Graph Representation Learning Graph Structured Data Vertical Federated Learning GNN Training Transaction Network

August 25, 2023

Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction
Guangji Bai, Ziyang Yu, Zheng Chai, Yue Cheng, Liang Zhao
Graph Neural Network Online Service Large Scale Graph GNN Training Large Scale Graph Datasets Dynamic Embeddings Stale Gradient GNN Training Framework

August 23, 2023

Cached Operator Reordering: A Unified View for Fast GNN Training
Julia Bazinska, Andrei Ivanov, Tal Ben-Nun, Nikoli Dryden, Maciej Besta, Siyuan Shen, Torsten Hoefler
Graph Neural Network Graph Convolutional Network Unified View GNN Training GNN Layer GNN Computation Large Scale GNN

August 19, 2023

The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive field
Kun Wang, Guohao Li, Shilong Wang, Guibin Zhang, Kai Wang, Yang You, Xiaojiang Peng, Yuxuan Liang, Yang Wang
Graph Neural Network Receptive Field Depth Network GNN Training GNN Backbone

August 6, 2023

Communication-Free Distributed GNN Training with Vertex Cut
Kaidi Cao, Rui Deng, Shirley Wu, Edward W Huang, Karthik Subbian, Jure Leskovec
GNN Training Limited Communication GNN Training Framework Normalized Cut

August 2, 2023

Tango: rethinking quantization for graph neural network training on GPUs
Shiyang Chen, Da Zheng, Caiwen Ding, Chengying Huan, Yuede Ji, Hang Liu
Neural Network Graph Neural Network Graph Drawing Single GPU Multiplier Free Quantization GNN Training GNN Computation

GNN Training

Papers

Efficient Message Passing Architecture for GCN Training on HBM-based FPGAs with Orthogonal Topology On-Chip Networks

Using Half-Precision for GNN Training

TCGU: Data-centric Graph Unlearning based on Transferable Condensation

Haste Makes Waste: A Simple Approach for Scaling Graph Neural Networks

FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale

HGNAS: Hardware-Aware Graph Neural Architecture Search for Edge Devices

Slicing Input Features to Accelerate Deep Learning: A Case Study with Graph Neural Networks

Distributed Training of Large Graph Neural Networks with Variable Communication Rates

Reducing Memory Contention and I/O Congestion for Disk-based GNN Training

Disttack: Graph Adversarial Attacks Toward Distributed GNN Training

DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Training

CATGNN: Cost-Efficient and Scalable Distributed Training for Graph Neural Networks

Task-Oriented GNNs Training on Large Knowledge Graphs for Accurate and Efficient Modeling

NeutronOrch: Rethinking Sample-based GNN Training under CPU-GPU Heterogeneous Environments

Privacy-preserving design of graph neural networks with applications to vertical federated learning

Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction

Cached Operator Reordering: A Unified View for Fast GNN Training

The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive field

Communication-Free Distributed GNN Training with Vertex Cut

Tango: rethinking quantization for graph neural network training on GPUs