Sparse Connectivity

Sparse connectivity, the strategic reduction of connections in networks, aims to improve efficiency and robustness while maintaining performance. Current research focuses on optimizing sparse architectures in neural networks (using techniques like pruning and attention mechanisms), analyzing the impact of sparse connectivity on network dynamics and learning, and developing algorithms for efficient sparse connectivity search and restoration in various applications, including robotics and graph-based models. This research is significant because sparse connectivity offers substantial benefits in terms of computational cost, memory usage, and generalization ability, impacting fields ranging from artificial intelligence to network science and beyond.

Papers

June 21, 2023

Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Training
Aleksandra I. Nowak, Bram Grooten, Decebal Constantin Mocanu, Jacek Tabor
Edge Pruning Balancing Weight Sparse Connectivity Dynamic Sparse Training Magnitude Based Pruning

June 12, 2023

A Graph Transformer-Driven Approach for Network Robustness Learning
Yu Zhang, Jia Li, Jie Ding, Xiang Li
Graph Drawing Sparse Connectivity Network Robustness

June 6, 2023

March 31, 2023

Directional Connectivity-based Segmentation of Medical Images
Ziyun Yang, Sina Farsiu
Medical Image Sparse Connectivity Consistent Segmentation Medical Image Segmentation Benchmark

February 22, 2023

Approximate spectral clustering density-based similarity for noisy datasets
Mashaan Alshammari, Masahiro Takatsuka
Spectral Clustering Sparse Connectivity Real World Noisy Datasets Graph Edge Cluster Shape

February 6, 2023

Connectivity Enhanced Safe Neural Network Planner for Lane Changing in Mixed Traffic
Xiangguo Liu, Ruochen Jiao, Bowen Zheng, Dave Liang, Qi Zhu
Autonomous Vehicle Sparse Connectivity Mixed Traffic Neural Planner Connected Vehicle Geometrically Sound LANE

February 3, 2023

LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
Jian Cheng Wong, Pao-Hsiung Chiu, Chinchun Ooi, My Ha Dao, Yew-Soon Ong
Physic Informed Neural Network Complex Valued Sparse Connectivity Complex Dynamic Physic Informed Learning Local Gradient Novel Loss

October 14, 2022

Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization
Kun Yuan, Xinmeng Huang, Yiming Chen, Xiaohan Zhang, Yingya Zhang, Pan Pan
Sparse Connectivity Non Convex Loss Function Decentralized Optimization Decentralized Algorithm Optimal Convergence Rate Decentralized Nonconvex

September 28, 2022

Consensus Knowledge Graph Learning via Multi-view Sparse Low Rank Block Model
Tianxi Cai, Dong Xia, Luwan Zhang, Doudou Zhou
Knowledge Graph Consensus Group Decision Sparse Connectivity Network Analysis

September 23, 2022

Multivariate Wasserstein Functional Connectivity for Autism Screening
Oleg Kachan, Alexander Bernstein
Functional Magnetic Resonance Imaging Autism Spectrum Disorder Functional Connectivity Sparse Connectivity

September 14, 2022

Optimizing Connectivity through Network Gradients for Restricted Boltzmann Machines
A. C. N. de Oliveira, D. R. Figueiredo
Sparse Network Shallow Network Sparse Connectivity Restricted Boltzmann Machine Massive Connectivity Network Gradient

August 7, 2022

N2NSkip: Learning Highly Sparse Networks using Neuron-to-Neuron Skip Connections
Arvind Subramaniam, Avinash Sharma
Pruning Method Sparse Network Sparse Connectivity Iterative Pruning Skip Connection DNN Pruning NN2Poly Package

June 28, 2022

Classification of ADHD Patients Using Kernel Hierarchical Extreme Learning Machine
Sartaj Ahmed Salman, Zhichao Lian, Milad Taleby Ahvanooey, Hiroki Takahashi, Yuduo Zhang
Classification Code Functional Connectivity Sparse Connectivity Brain Dynamic Brain Imaging Data Attention Deficit Hyperactivity Disorder Kernel Extreme Learning Machine

May 17, 2022

CAS-Net: Conditional Atlas Generation and Brain Segmentation for Fetal MRI
Liu Li, Qiang Ma, Matthew Sinclair, Antonios Makropoulos, Joseph Hajnal, A. David Edwards, Bernhard Kainz, Daniel Rueckert, Amir Alansary
Sparse Connectivity Human Connectome Project Brain Segmentation Fetal MRI Brain Tissue Segmentation Brain Tissue Conditional Atlas

May 11, 2022

NDGGNET-A Node Independent Gate based Graph Neural Networks
Ye Tang, Xuesong Yang, Xinrui Liu, Xiwei Zhao, Zhangang Lin, Changping Peng
Graph Neural Network GNN Model Sparse Connectivity Gated Recurrent

March 27, 2022

Distributed Link Sparsification for Scalable Scheduling Using Graph Neural Networks
Zhongyuan Zhao, Ananthram Swami, Santiago Segarra
Graph Neural Network Large Scale Sparse Connectivity Network Sparsity Aware Job

March 23, 2022

DPST: De Novo Peptide Sequencing with Amino-Acid-Aware Transformers
Yan Yang, Zakir Hossain, Khandaker Asif, Liyuan Pan, Shafin Rahman, Eric Stone
Sparse Connectivity Amino Acid Mass Spectrometry Protein Transformer

March 20, 2022

A Learning Convolutional Neural Network Approach for Network Robustness Prediction
Yang Lou, Ruizi Wu, Junli Li, Lin Wang, Xiang Li, Guanrong Chen
Convolutional Neural Network Native Robustness Sparse Connectivity Network Robustness

March 16, 2022

Tangles and Hierarchical Clustering
Eva Fluck
Submodular Maximization Hierarchical Clustering Sparse Connectivity Clustering Result

Sparse Connectivity

Papers

Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Training

A Graph Transformer-Driven Approach for Network Robustness Learning

Towards Memory-Efficient Training for Extremely Large Output Spaces -- Learning with 500k Labels on a Single Commodity GPU

ESL-SNNs: An Evolutionary Structure Learning Strategy for Spiking Neural Networks

Directional Connectivity-based Segmentation of Medical Images

Approximate spectral clustering density-based similarity for noisy datasets

Connectivity Enhanced Safe Neural Network Planner for Lane Changing in Mixed Traffic

LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry

Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization

Consensus Knowledge Graph Learning via Multi-view Sparse Low Rank Block Model

Multivariate Wasserstein Functional Connectivity for Autism Screening

Optimizing Connectivity through Network Gradients for Restricted Boltzmann Machines

N2NSkip: Learning Highly Sparse Networks using Neuron-to-Neuron Skip Connections

Classification of ADHD Patients Using Kernel Hierarchical Extreme Learning Machine

CAS-Net: Conditional Atlas Generation and Brain Segmentation for Fetal MRI

NDGGNET-A Node Independent Gate based Graph Neural Networks

Distributed Link Sparsification for Scalable Scheduling Using Graph Neural Networks

DPST: De Novo Peptide Sequencing with Amino-Acid-Aware Transformers

A Learning Convolutional Neural Network Approach for Network Robustness Prediction

Tangles and Hierarchical Clustering