Selective Neural Network

Selective neural networks aim to improve efficiency and robustness of deep learning models by strategically focusing computation or training on subsets of data or model parameters. Current research explores various selective mechanisms, including selective backpropagation for faster training, selective feature extraction and fusion for enhanced performance in tasks like speech coding and medical image analysis, and selective prediction to improve model reliability by abstaining from uncertain predictions. These advancements are significant because they address computational bottlenecks in training large models, improve the accuracy and efficiency of various applications, and enhance the reliability of predictions in critical domains such as healthcare and autonomous driving.

Papers

September 12, 2024

Microscopic-Mamba: Revealing the Secrets of Microscopic Images with Just 4M Parameters
Shun Zou, Zhuo Zhang, Yi Zou, Guangwei Gao
CNN Model Medical Image Classification Local Feature Feature Interaction Microscopic Image Dark Secret Selective Neural Network

July 30, 2024

SuperCodec: A Neural Speech Codec with Selective Back-Projection Network
Youqiang Zheng, Weiping Tu, Li Xiao, Xinmeng Xu
Neural Speech Selective Neural Network

December 8, 2023

A Negative Result on Gradient Matching for Selective Backprop
Lukas Balles, Cedric Archambeau, Giovanni Zappella
Subset Selection Forward Pas Gradient Matching Selective Neural Network Random Baseline

August 21, 2023

Label Selection Approach to Learning from Crowds
Kosuke Yoshimura, Hisashi Kashima
Crowded Environment Scale Crowd Network Selection Crowd Annotation Selective Neural Network Crowd Model

April 25, 2023

Time-Selective RNN for Device-Free Multi-Room Human Presence Detection Using WiFi CSI
Li-Hsiang Shen, An-Hung Hsiao, Fang-Yu Chu, Kai-Ten Feng
Channel State Information Dependent Feature Selective Neural Network Human Presence Detection

November 29, 2022

An Extreme-Adaptive Time Series Prediction Model Based on Probability-Enhanced LSTM Neural Networks
Yanhong Li, Jack Xu, David C. Anastasiu
Probabilistic Neural Network Adaptive Forecasting Selective Neural Network Prediction Function

November 19, 2022

Gumbel-Softmax Selective Networks
Mahmoud Salem, Mohamed Osama Ahmed, Frederick Tung, Gabriel Oliveira
Gumbel Softmax Selective Neural Network Selective Regression

July 27, 2022

PointFix: Learning to Fix Domain Bias for Robust Online Stereo Adaptation
Kwonyoung Kim, Jungin Park, Jiyoung Lee, Dongbo Min, Kwanghoon Sohn
LeArning Abstract Stereo Matching Robust Initialization Domain Bias Base Model Selective Neural Network

May 26, 2022

Selective Classification Via Neural Network Training Dynamics
Stephan Rabanser, Anvith Thudi, Kimia Hamidieh, Adam Dziedzic, Nicolas Papernot
Model Accuracy Selective Classification Selective Neural Network

January 25, 2022

Representations learnt by SGD and Adaptive learning rules: Conditions that vary sparsity and selectivity in neural networks
Jin Hyun Park
Neural Network Stochastic Gradient Descent Meaningful Representation Sparsity Increase Sparse Activation Selective Engagement Learning Rule Selective Neural Network

January 21, 2022

Contrastive and Selective Hidden Embeddings for Medical Image Segmentation
Zhuowei Li, Zihao Liu, Zhiqiang Hu, Qing Xia, Ruiqin Xiong, Shaoting Zhang, Dimitris Metaxas, Tingting Jiang
Contrastive Learning Medical Image Segmentation Medical Image Segmentation Task Contrastive Pretraining Contrastive Regularization Selective Neural Network