Squashing Mitigation Approach

Squashing mitigation in graph neural networks (GNNs) focuses on addressing the limitations of message-passing architectures in capturing long-range dependencies within graph data. Current research investigates phenomena like over-squashing (loss of distant node information) and over-smoothing (feature homogenization), employing techniques such as graph rewiring, novel normalization methods, and curvature-based approaches to improve information flow and expressivity. These efforts aim to enhance the performance of GNNs on tasks requiring the integration of global context, impacting diverse fields from social network analysis to drug discovery.

Papers

October 7, 2024

Taming Gradient Oversmoothing and Expansion in Graph Neural Networks
MoonJeong Park, Dongwoo Kim
Graph Neural Network Gradient Flow Prompt Expansion Gradient Approximation Squashing Mitigation Approach Residual Graph Convolutional Network

May 22, 2024

Understanding Virtual Nodes: Oversmoothing, Oversquashing, and Node Heterogeneity
Joshua Southern, Francesco Di Giovanni, Michael Bronstein, Johannes F. Lutzeyer
Random Node Message Passing Neural Network Graph Level Task Virtual Network Squashing Mitigation Approach Graph Aware Transformer Network Heterogeneity

May 2, 2024

ATNPA: A Unified View of Oversmoothing Alleviation in Graph Neural Networks
Yufei Jin, Xingquan Zhu
Graph Neural Network Gene Level GNN Unified View Squashing Mitigation Approach Shallow Layer

April 6, 2024

Spectral Graph Pruning Against Over-Squashing and Over-Smoothing
Adarsh Jamadandi, Celia Rubio-Madrigal, Rebekka Burkholz
Message Passing Graph Neural Network Undisciplined Over Smoothing Squashing Mitigation Approach Spectral Gap

January 23, 2024

DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing
Li Sun, Zhenhao Huang, Hua Wu, Junda Ye, Hao Peng, Zhengtao Yu, Philip S. Yu
Graph Neural Network Graph Structure Graph Structure Learning Squashing Mitigation Approach Riemannian Graph

January 9, 2024

Setting the Record Straight on Transformer Oversmoothing
Gbètondji J-S Dovonon, Michael M. Bronstein, Matt J. Kusner
Strong Generalization Deeper Model Squashing Mitigation Approach Fact Extraction Music Recording Transformer Based Structure Transformer Prediction

December 27, 2023

Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreaching
Federico Errica, Henrik Christiansen, Viktor Zaverkin, Takashi Maruyama, Mathias Niepert, Francesco Alesiani
Variational Inference General Framework Long Range Long Range Dependency Long Range Interaction Squashing Mitigation Approach Approximate Message Passing

December 14, 2023

Overdrawing Urns using Categories of Signed Probabilities
Bart Jacobs, Dario Stein
Probabilistic Model Category Shift Squashing Mitigation Approach Cumulative Distribution Transform

November 27, 2023

Over-Squashing in Riemannian Graph Neural Networks
Julia Balla
Riemannian Manifold Gaussian Curvature Squashing Mitigation Approach Riemannian Graph Hyperbolic Graph Variable Curvature

November 13, 2023

Exposition on over-squashing problem on GNNs: Current Methods, Benchmarks and Challenges
Dai Shi, Andi Han, Lequan Lin, Yi Guo, Junbin Gao
New Benchmark Technical Challenge Gene Level GNN Right Problem Message Passing Neural Network Message Passing Graph Neural Network Current Method Graph Node Squashing Mitigation Approach Graph Level Learning

September 17, 2023

Mitigating Over-Smoothing and Over-Squashing using Augmentations of Forman-Ricci Curvature
Lukas Fesser, Melanie Weber
Graph Neural Network Soft Augmentation Large Scale Graph Message Passing Graph Neural Network Squashing Mitigation Approach Smoothing Problem Ollivier Ricci Curvature

August 29, 2023

Over-Squashing in Graph Neural Networks: A Comprehensive survey
Singh Akansha
Graph Neural Network Comprehensive Survey Graph Structured Data Graph Level Task Squashing Mitigation Approach Graph Rewiring

June 6, 2023

How does over-squashing affect the power of GNNs?
Francesco Di Giovanni, T. Konstantin Rusch, Michael M. Bronstein, Andreea Deac, Marc Lackenby, Siddhartha Mishra, Petar Veličković
Graph Neural Network Real Power Message Passing Neural Network Input Graph Squashing Mitigation Approach

February 6, 2023

On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology
Francesco Di Giovanni, Lorenzo Giusti, Federico Barbero, Giulia Luise, Pietro Lio', Michael Bronstein
Graph Neural Network Large Depth Message Passing Neural Network Topology Problem Squashing Mitigation Approach Influential Node Minimum Width Width Neural Network

December 13, 2022

Leave Graphs Alone: Addressing Over-Squashing without Rewiring
Domenico Tortorella, Alessio Micheli
Graph Drawing Reservoir Computing Message Passing Graph Information Bottleneck Squashing Mitigation Approach Graph Connectivity Graph Echo State Network

November 28, 2022

Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature
Khang Nguyen, Hieu Nong, Vinh Nguyen, Nhat Ho, Stanley Osher, Tan Nguyen
Graph Neural Network Interaction Graph Squashing Mitigation Approach Smoothing Problem Local Geometry Graph Curvature Ollivier Ricci Curvature

September 23, 2022

Overtwisting and Coiling Highly Enhances Strain Generation of Twisted String Actuators
Revanth Konda, David Bombara, Jun Zhang
Squashing Mitigation Approach Multi Coil Strain Generation

March 9, 2022

Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice
Peihao Wang, Wenqing Zheng, Tianlong Chen, Zhangyang Wang
Convolutional Neural Network Vision Transformer Theoretical Understanding Deep Transformer Squashing Mitigation Approach Self Attention Block Self Attention Matrix

January 22, 2022

Overcoming Oversmoothness in Graph Convolutional Networks via Hybrid Scattering Networks
Frederik Wenkel, Yimeng Min, Matthew Hirn, Michael Perlmutter, Guy Wolf
Convolutional Neural Network Graph Structured Data Graph Signal Scattering Network Squashing Mitigation Approach

November 29, 2021

Understanding over-squashing and bottlenecks on graphs via curvature
Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, Michael M. Bronstein
Graph Drawing Gaussian Curvature Major Challenge Bottleneck Graph Information Bottleneck Squashing Mitigation Approach Graph Curvature Hop Neighbor