Convolutional Counterpart

Convolutional counterparts, primarily convolutional neural networks (CNNs), are being actively investigated for improvements in efficiency and robustness. Research focuses on optimizing CNN architectures through techniques like depth compression, dynamic convolution (adapting kernel weights based on input), and novel weight initialization methods tailored to tensorial CNNs. These efforts aim to enhance performance metrics such as accuracy and inference speed while reducing computational cost and energy consumption, impacting various applications from image recognition and synthesis to object detection. Furthermore, comparisons with Vision Transformers (ViTs) highlight strengths and weaknesses of each approach, driving the development of more robust and efficient models.

Papers

October 5, 2023

Fishnets: Information-Optimal, Scalable Aggregation for Sets and Graphs
T. Lucas Makinen, Justin Alsing, Benjamin D. Wandelt
Graph Neural Network Graph Drawing Tiny Set Robust Aggregation Convolutional Counterpart Optimal Embeddings

February 7, 2023

Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs
Saro Passaro, C. Lawrence Zitnick
Direct Convolution Equivariant Network Order Tensor Equivariant Convolution Convolutional Counterpart

January 28, 2023

Efficient Latency-Aware CNN Depth Compression via Two-Stage Dynamic Programming
Jinuk Kim, Yeonwoo Jeong, Deokjae Lee, Hyun Oh Song
Convolution Layer Neural Network Compression Convolution Operation Continuous Convolution Deep Compression Convolutional Counterpart

December 13, 2022

What do Vision Transformers Learn? A Visual Exploration
Amin Ghiasi, Hamid Kazemi, Eitan Borgnia, Steven Reich, Manli Shu, Micah Goldblum, Andrew Gordon Wilson, Tom Goldstein
Convolutional Neural Network Vision Transformer Computer Vision Visual Feature Visual Exploration Convolutional Counterpart

September 16, 2022

Omni-Dimensional Dynamic Convolution
Chao Li, Aojun Zhou, Anbang Yao
Convolutional Kernel Dynamic Convolution Convolutional Counterpart

June 23, 2022

Universal Learned Image Compression With Low Computational Cost
Bowen Li, Yao Xin, Youneng Bao, Fanyang Meng, Yongsheng Liang, Wen Tan
Image Compression Dynamic ModulE Convolutional Neural Network Architecture Convolutional Counterpart

May 28, 2022

A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks
Yu Pan, Zeyong Su, Ao Liu, Jingquan Wang, Nannan Li, Zenglin Xu
Weight Initialization Convolutional Counterpart

February 24, 2022

Auto-scaling Vision Transformers without Training
Wuyang Chen, Wei Huang, Xianzhi Du, Xiaodan Song, Zhangyang Wang, Denny Zhou
Vision Transformer ImageNet 1k Auto Scaling Convolutional Counterpart ViT Architecture

December 29, 2021

Learning Spatially-Adaptive Squeeze-Excitation Networks for Image Synthesis and Image Recognition
Jianghao Shen, Tianfu Wu
Image Synthesis Image Recognition Convolution Layer Multi Head Self Attention Pyramid Transformer Convolutional Counterpart Squeeze Excitation Squeeze and Excitation

December 27, 2021

An Empirical Study of Adder Neural Networks for Object Detection
Xinghao Chen, Chang Xu, Minjing Dong, Chunjing Xu, Yunhe Wang
Empirical Study Pre Trained Backbone Traditional Convolutional Convolutional Counterpart Adder Neural Network

November 20, 2021

Discrete Representations Strengthen Vision Transformer Robustness
Chengzhi Mao, Lu Jiang, Mostafa Dehghani, Carl Vondrick, Rahul Sukthankar, Irfan Essa
Vision Transformer Supervised ImageNet Image Recognition Convolutional Counterpart