Convolutional Architecture

Convolutional architectures are a cornerstone of deep learning, primarily used for processing grid-like data such as images and audio, aiming to efficiently extract hierarchical features through convolutional operations. Current research focuses on improving efficiency (e.g., through pruning, quantization, and novel architectures like CondenseNet), enhancing robustness (especially to occlusion), and exploring alternatives or augmentations to convolutional layers (e.g., incorporating self-attention mechanisms from Transformers or using spiking neural networks). These advancements are driving progress in diverse applications, including medical image analysis, robotics, and automatic modulation classification, by improving accuracy, reducing computational costs, and enhancing model interpretability.

Papers

October 9, 2024

Exploring the design space of deep-learning-based weather forecasting systems
Shoaib Ahmed Siddiqui, Jean Kossaifi, Boris Bonev, Christopher Choy, Jan Kautz, David Krueger, Kamyar Azizzadenesheli
Deep Learning Design Space Convolutional Architecture Grid Based

September 18, 2024

Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers
Ekaterina Grishina, Mikhail Gorbunov, Maxim Rakhuba
Convolutional Layer Convolution Kernel Upper Bound Convolution Operation Convolutional Architecture Spectral Norm

September 16, 2024

Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?
Kaleb Kassaw, Francesco Luzi, Leslie M. Collins, Jordan M. Malof
Convolutional Neural Network Vision Transformer Recognition Task Robust Deep Convolutional Architecture

August 22, 2024

Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers
Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Reduan Achtibat, Thomas Wiegand, Wojciech Samek, Sebastian Lapuschkin
Neural Network Convolutional Neural Network Transformer Megatron Decepticons Line by Line Explanation Model Compression Edge Pruning Attribution Method Transformer Based Network Neural Network Pruning Convolutional Architecture

June 14, 2024

Comparison of fine-tuning strategies for transfer learning in medical image classification
Ana Davila, Jacinto Colan, Yasuhisa Hasegawa
Fine Tuning Transfer Learning Consistent Comparison Medical Imaging Medical Image Classification Convolutional Architecture Fine Tuning Strategy

May 4, 2024

Scaling SNNs Trained Using Equilibrium Propagation to Convolutional Architectures
Jiaqi Lin, Malyaban Bal, Abhronil Sengupta
Recurrent Neural Network Spiking Neural Network Recurrent Network Convolutional Architecture Equilibrium Propagation Non Spiking

April 18, 2024

When Medical Imaging Met Self-Attention: A Love Story That Didn't Quite Work Out
Tristan Piater, Niklas Penzel, Gideon Stein, Joachim Denzler
Deep Convolutional Neural Network Self Attention Convolutional Architecture

December 19, 2023

SimQ-NAS: Simultaneous Quantization Policy and Neural Architecture Search
Sharath Nittur Sridhar, Maciej Szankin, Fang Chen, Sairam Sundaresan, Anthony Sarah
Convolutional Neural Network Neural Architecture Search Transformer Based Architecture Convolutional Architecture One Shot Neural Architecture Search Quantization Search

August 22, 2023

Exemplar-Free Continual Transformer with Convolutions
Anurag Roy, Vinay Kumar Verma, Sravan Voonna, Kripabandhu Ghosh, Saptarshi Ghosh, Abir Das
Vision Transformer Continual LEArning Incremental Learning Direct Convolution Convolutional Architecture Free Continual Learning Exemplar Free Method

August 3, 2023

A Novel Convolutional Neural Network Architecture with a Continuous Symmetry
Yao Liu, Hang Shao, Bing Bai
Neural Network Convolutional Neural Network High Performing 1D ConvNet Convolutional Architecture Continuous Symmetry

July 21, 2023

Local Kernel Renormalization as a mechanism for feature learning in overparametrized Convolutional Neural Networks
R. Aiudi, R. Pacelli, A. Vezzani, R. Burioni, P. Rotondo
Convolutional Neural Network Feature Learning Functional Mechanism Convolutional Architecture Fully Connected Neural Network Width Network

July 7, 2023

RCDN -- Robust X-Corner Detection Algorithm based on Advanced CNN Model
Ben Chen, Caihua Xiong, Quanlin Li, Zhonghua Wan
Computer Vision Accurate Detection Convolutional Architecture Corner Detection X Corner

July 2, 2023

Conformer LLMs -- Convolution Augmented Large Language Models
Prateek Verma
Large Language Model Language Model Transformer Decoder Convolutional Architecture Causal Convolution Robust Speech

June 21, 2023

Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
Boyan Li, Luziwei Leng, Shuaijie Shen, Kaixuan Zhang, Jianguo Zhang, Jianxing Liao, Ran Cheng
Efficient Deep Convolutional Architecture Spike Based MLP Architecture

June 16, 2023

Systematic Architectural Design of Scale Transformed Attention Condenser DNNs via Multi-Scale Class Representational Response Similarity Analysis
Andre Hryniowski, Alexander Wong
Self Attention Self Attention Module Convolutional Architecture Local Similarity Architectural Design Attention Condenser

May 29, 2023

Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks
Haotian Jiang, Qianxiao Li
Temporal Convolutional Network Convolutional Architecture Temporal Sequence Approximation Rate

May 9, 2023

Regular Splitting Graph Network for 3D Human Pose Estimation
Tanvir Hassan, A. Ben Hamza
Pose Estimation Human Pose Estimation 3D Human Pose Estimation Convolutional Architecture Adjacency Hopping

April 3, 2023

Transformer-based interpretable multi-modal data fusion for skin lesion classification
Theodor Cheslerean-Boghiu, Melia-Evelina Fleischmann, Theresa Willem, Tobias Lasser
Deep Learning Multi Modal Data Skin Lesion Convolutional Architecture

February 10, 2023

DNArch: Learning Convolutional Neural Architectures by Backpropagation
David W. Romero, Neil Zeghidour
Convolutional Neural Network Back Propagation Neural Architecture Convolutional Architecture Differentiable Neural Patch Masking Performant Neural Network

January 27, 2023

Automatic Modulation Classification with Deep Neural Networks
Clayton Harper, Mitchell Thornton, Eric Larson
Deep Neural Network Convolutional Architecture Automatic Modulation Classification Software Defined Radio

Convolutional Architecture

Papers

Exploring the design space of deep-learning-based weather forecasting systems

Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers

Are Deep Learning Models Robust to Partial Object Occlusion in Visual Recognition Tasks?

Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers

Comparison of fine-tuning strategies for transfer learning in medical image classification

Scaling SNNs Trained Using Equilibrium Propagation to Convolutional Architectures

When Medical Imaging Met Self-Attention: A Love Story That Didn't Quite Work Out

SimQ-NAS: Simultaneous Quantization Policy and Neural Architecture Search

Exemplar-Free Continual Transformer with Convolutions

A Novel Convolutional Neural Network Architecture with a Continuous Symmetry

Local Kernel Renormalization as a mechanism for feature learning in overparametrized Convolutional Neural Networks

RCDN -- Robust X-Corner Detection Algorithm based on Advanced CNN Model

Conformer LLMs -- Convolution Augmented Large Language Models

Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference

Systematic Architectural Design of Scale Transformed Attention Condenser DNNs via Multi-Scale Class Representational Response Similarity Analysis

Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks

Regular Splitting Graph Network for 3D Human Pose Estimation

Transformer-based interpretable multi-modal data fusion for skin lesion classification

DNArch: Learning Convolutional Neural Architectures by Backpropagation

Automatic Modulation Classification with Deep Neural Networks