Undisciplined Over Smoothing

Undisciplined over-smoothing in graph neural networks (GNNs) refers to the phenomenon where, as network depth increases, node representations become increasingly similar, hindering performance. Current research focuses on mitigating this issue through various techniques, including modified aggregation functions, architectural innovations like residual connections and skip connections, and the development of novel normalization methods. Addressing over-smoothing is crucial for realizing the full potential of deep GNNs across diverse applications, from graph classification to natural language processing and image analysis, enabling more accurate and robust models for complex data.

Papers

August 9, 2024

Better Not to Propagate: Understanding Edge Uncertainty and Over-smoothing in Signed Graph Neural Networks
Yoonhyuk Choi, Jiho Choi, Taewook Ko, Chong-Kwon Kim
Graph Neural Network Heterophilic Graph Propagation Environment Undisciplined Over Smoothing Linear Separability

August 7, 2024

Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks
Jie Peng, Runlin Lei, Zhewei Wei
Graph Neural Network Undisciplined Over Smoothing Deep Graph Model

August 6, 2024

TSC: A Simple Two-Sided Constraint against Over-Smoothing
Furong Peng, Kang Liu, Xuan Lu, Yuhua Qian, Hongren Yan, Chao Ma
Graph Convolutional Network Node Representation Deeper Network Graph Convolutional Neural Network Undisciplined Over Smoothing Order Neighbor

July 27, 2024

Alleviating Over-Smoothing via Aggregation over Compact Manifolds
Dongzhuoran Zhou, Hui Yang, Bo Xiong, Yue Ma, Evgeny Kharlamov
Graph Neural Network Graph Drawing Class Manifold Data Aggregation Undisciplined Over Smoothing Information Aggregation

July 16, 2024

Simplifying the Theory on Over-Smoothing
Andreas Roth
Theoretical Understanding Graph Convolution Undisciplined Over Smoothing Rank Collapse Power Iteration

July 1, 2024

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks
Guangrui Yang, Jianfei Li, Ming Li, Han Feng, Ding-Xuan Zhou
Graph Neural Network Average Approximation Theoretical Analysis Undisciplined Over Smoothing Approximation Theory Average Smoothness Approximation Property Approximation Capability

May 21, 2024

How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing
Keke Huang, Yu Guang Wang, Ming Li, and Pietro Liò
Heterophilic Graph Graph Filter Spectral Graph Neural Network Undisciplined Over Smoothing Polynomial Filter Polynomial Basis

April 6, 2024

Spectral Graph Pruning Against Over-Squashing and Over-Smoothing
Adarsh Jamadandi, Celia Rubio-Madrigal, Rebekka Burkholz
Message Passing Graph Neural Network Undisciplined Over Smoothing Squashing Mitigation Approach Spectral Gap

April 3, 2024

First-order PDES for Graph Neural Networks: Advection And Burgers Equation Models
Yifan Qu, Oliver Krzysik, Hans De Sterck, Omer Ege Kara
Graph Neural Network Undisciplined Over Smoothing Implicit Differentiation Smoothing Problem Advection Diffusion Process Burger Equation

December 13, 2023

Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs
Jin Li, Qirong Zhang, Shuling Xu, Xinlong Chen, Longkun Guo, Yang-Geng Fu
Deep Network Node Embeddings Undisciplined Over Smoothing Input Graph

November 6, 2023

From Coupled Oscillators to Graph Neural Networks: Reducing Over-smoothing via a Kuramoto Model-based Approach
Tuan Nguyen, Hirotada Honda, Takashi Sano, Vinh Nguyen, Shugo Nakamura, Tan M. Nguyen
Graph Neural Network Undisciplined Over Smoothing Kuramoto Model Coupled Oscillator

September 6, 2023

Unifying over-smoothing and over-squashing in graph neural networks: A physics informed approach and beyond
Zhiqi Shao, Dai Shi, Andi Han, Yi Guo, Qibin Zhao, Junbin Gao
Neural Network Graph Neural Network Graph Drawing Constructive Approach Theoretical Physic Graph Node Undisciplined Over Smoothing Node Feature Sharpness Heat Kernel Heat Equation

August 31, 2023

August 7, 2023

Implicit Graph Neural Diffusion Networks: Convergence, Generalization, and Over-Smoothing
Guoji Fu, Mohammed Haroon Dupty, Yanfei Dong, Lee Wee Sun
Strong Generalization Early Stage Convergence Graph Neural Graph Laplacian Implicit Neural Network Undisciplined Over Smoothing Implicit Graph Neural Network

May 9, 2023

Alleviating Over-smoothing for Unsupervised Sentence Representation
Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Bowen Cao, Jianhui Chang, Daxin Jiang, Jia Li
Contrastive Learning Pre Trained Language Model Undisciplined Over Smoothing Self Contrastive Unsupervised Sentence Representation

May 4, 2023

Mitigating Undisciplined Over-Smoothing in Transformer for Weakly Supervised Semantic Segmentation
Jingxuan He, Lechao Cheng, Chaowei Fang, Dingwen Zhang, Zhangye Wang, Wei Chen
Semantic Segmentation Transformer Based Attention Map Weakly Supervised Semantic Segmentation Segmentation Result Undisciplined Over Smoothing Affinity Matrix

March 20, 2023

A Survey on Oversmoothing in Graph Neural Networks
T. Konstantin Rusch, Michael M. Bronstein, Siddhartha Mishra
Graph Neural Network Timely Survey Graph Learning Task Undisciplined Over Smoothing Smoothing Problem

February 3, 2023

Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing
Yunchong Song, Chenghu Zhou, Xinbing Wang, Zhouhan Lin
Graph Drawing Node Representation Message Passing Undisciplined Over Smoothing System Message Explainable Graph

January 7, 2023

Reducing Over-smoothing in Graph Neural Networks Using Relational Embeddings
Yeskendir Koishekenov
Graph Neural Network Graph Attention Network Graph Node Undisciplined Over Smoothing Relational Embeddings GNN Application

Undisciplined Over Smoothing

Papers

Better Not to Propagate: Understanding Edge Uncertainty and Over-smoothing in Signed Graph Neural Networks

Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks

TSC: A Simple Two-Sided Constraint against Over-Smoothing

Alleviating Over-Smoothing via Aggregation over Compact Manifolds

Simplifying the Theory on Over-Smoothing

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks

How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing

Spectral Graph Pruning Against Over-Squashing and Over-Smoothing

First-order PDES for Graph Neural Networks: Advection And Burgers Equation Models

Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs

From Coupled Oscillators to Graph Neural Networks: Reducing Over-smoothing via a Kuramoto Model-based Approach

Unifying over-smoothing and over-squashing in graph neural networks: A physics informed approach and beyond

Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networks

Curvature-based Pooling within Graph Neural Networks

Implicit Graph Neural Diffusion Networks: Convergence, Generalization, and Over-Smoothing

Alleviating Over-smoothing for Unsupervised Sentence Representation

Mitigating Undisciplined Over-Smoothing in Transformer for Weakly Supervised Semantic Segmentation

A Survey on Oversmoothing in Graph Neural Networks

Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing

Reducing Over-smoothing in Graph Neural Networks Using Relational Embeddings