Variational Bottleneck

Variational bottlenecks are a technique used in deep learning to constrain the information flow within neural networks, aiming to improve efficiency, privacy, and model interpretability. Current research focuses on applying variational bottlenecks in diverse architectures, including convolutional neural networks (CNNs) and generative adversarial networks (GANs), to achieve goals such as real-time object detection on resource-constrained devices, enhanced privacy in federated learning, and efficient feature compression for mobile edge computing. This approach offers significant potential for advancing various fields, from improving the performance and security of AI systems to enabling more reliable and interpretable medical image analysis.

Papers

July 29, 2024

Octave-YOLO: Cross frequency detection network with octave convolution
Sangjune Shin, Dongkun Shin
Real Time Object Convolutional Neural Network Architecture Spatial Frequency Deep Learning Task Variational Bottleneck

February 12, 2024

Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning
Yuxiao Wen, Arthur Jacot
CNN Model Feature Learning Major Challenge Bottleneck Large Learning Rate Dominant Low Frequency Variational Bottleneck Bottleneck Structure

September 8, 2023

Privacy Preserving Federated Learning with Convolutional Variational Bottlenecks
Daniel Scheliga, Patrick Mäder, Marco Seeland
Gradient Inversion Attack Privacy Preserving Federated Learning Gradient Leakage Gradient Leakage Attack Variational Bottleneck

February 21, 2023

FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing
Alireza Furutanpey, Philipp Raith, Schahram Dustdar
Deep Neural Network Mobile Edge Computing AI Accelerator Lightweight Deep Variational Bottleneck Variational Compression

February 6, 2023

LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval
Ziyang luo, Pu Zhao, Can Xu, Xiubo Geng, Tao Shen, Chongyang Tao, Jing Ma, Qingwen lin, Daxin Jiang
App to App Retrieval Dense Retrieval Language Image Lexicon Based Variational Bottleneck

December 31, 2022

Pseudo-Inverted Bottleneck Convolution for DARTS Search Space
Arash Ahmadian, Louis S. P. Liu, Yue Fei, Konstantinos N. Plataniotis, Mahdi S. Hosseini
Neural Architecture Search CNN Model Major Challenge Bottleneck Differentiable Architecture Search Variational Bottleneck

October 12, 2022

What can we learn about a generated image corrupting its latent representation?
Agnieszka Tomczak, Aarushi Gupta, Slobodan Ilic, Nassir Navab, Shadi Albarqouni
GAN Model Latent Representation Generative Question Image to Image Translation Problem Variational Bottleneck

June 5, 2022

U(1) Symmetry-breaking Observed in Generic CNN Bottleneck Layers
Louis-François Bouchard, Mohsen Ben Lazreg, Matthew Toews
Deep Convolutional Neural Network Supervised ImageNet Biological Vision Variational Bottleneck

May 13, 2022

From Images to Probabilistic Anatomical Shapes: A Deep Variational Bottleneck Approach
Jadie Adams, Shireen Elhabian
3D Medical Image Statistical Shape Shape Representation Anatomical Shape Variational Bottleneck

Variational Bottleneck

Papers

Octave-YOLO: Cross frequency detection network with octave convolution

Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning

Privacy Preserving Federated Learning with Convolutional Variational Bottlenecks

FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing

LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval

Pseudo-Inverted Bottleneck Convolution for DARTS Search Space

What can we learn about a generated image corrupting its latent representation?

U(1) Symmetry-breaking Observed in Generic CNN Bottleneck Layers

From Images to Probabilistic Anatomical Shapes: A Deep Variational Bottleneck Approach