Deep Architecture

Deep architectures, encompassing deep neural networks with numerous layers, aim to improve the accuracy and efficiency of machine learning models across diverse applications. Current research focuses on optimizing existing architectures like convolutional neural networks (CNNs) and transformers, exploring techniques such as model compression, early exiting, and novel training strategies to enhance performance and address limitations in resource-constrained environments. These advancements are significant for improving the efficiency and applicability of deep learning in areas like computer vision, natural language processing, and system identification, impacting both scientific understanding and practical deployment of AI systems.

Papers

October 15, 2024

Learning to rumble: Automated elephant call classification, detection and endpointing using deep architectures
Christiaan M. Geldenhuys, Thomas R. Niesler
Classification Code Data Detection Audio Spectrogram Transformer Deep Architecture Speech Detection Meta Classifier

October 10, 2024

From Logits to Hierarchies: Hierarchical Clustering made Simple
Emanuele Palumbo, Moritz Vandenhirtz, Alain Ryser, Imant Daunhawer, Julia E. Vogt
Pre Trained Part Whole Hierarchy Hierarchical Clustering Deep Architecture Second Ranked Logits Multi Stage Clustering

August 15, 2024

August 2, 2024

Transformers are Universal In-context Learners
Takashi Furuya, Maarten V. de Hoop, Gabriel Peyré
Vision Transformer Transformer Megatron Decepticons Context Learning Deep Transformer Deep Architecture

June 17, 2024

Just How Flexible are Neural Networks in Practice?
Ravid Shwartz-Ziv, Micah Goldblum, Arpit Bansal, C. Bayan Bruss, Yann LeCun, Andrew Gordon Wilson
Neural Network Convolutional Neural Network Practice Mode Deep Architecture Stochastic Learning Full Batch Gradient Descent

June 6, 2024

DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs
Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, Yu-Gang Jiang
Large Language Model Large Multimodal Model Visual Token Deep Architecture Deep Framework

June 5, 2024

Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers
Federico Bassetti, Marco Gherardi, Alessandro Ingrosso, Mauro Pastore, Pietro Rotondo
Deep Learning Convolutional Layer Structured Output Feature Learning Deep Linear Deep Architecture

December 26, 2023

Exploiting the capacity of deep networks only at training stage for nonlinear black-box system identification
Vahid MohammadZadeh Eivaghi, Mahdi Aliyari Shooredeli
Deep Network Deep Model Deep Generative Model System Identification Deep Architecture Capacity Loss Basis Function Training Stage

December 22, 2023

Training Neural Networks with Internal State, Unconstrained Connectivity, and Discrete Activations
Alexander Grushin
Neural Network Internal State Deep Architecture Training Algorithm Discrete Activation Binary Activation Connectivity Constraint

October 24, 2023

Automatic Aorta Segmentation with Heavily Augmented, High-Resolution 3-D ResUNet: Contribution to the SEG.A Challenge
Marek Wodzinski, Henning Müller
High Resolution U Net Encoder Decoder Client Contribution Deep Architecture Self Augmentation Quantitative Segmentation Aorta Segmentation

September 20, 2023

Hand Gesture Recognition with Two Stage Approach Using Transfer Learning and Deep Ensemble Learning
Serkan Savaş, Atilla Ergüzen
Convolutional Neural Network Transfer Learning Deep Ensemble Hand Gesture Recognition Deep Architecture Gesture Class Stage Approach

September 14, 2023

Towards a universal mechanism for successful deep learning
Yuval Meir, Yarden Tzach, Shiri Hodassman, Ofek Tevet, Ido Kanter
Deep Learning ImageNet Dataset Signal to Noise Ratio Deep Architecture General Purpose Model

August 31, 2023

Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting
Riccardo Miccini, Alaa Zniber, Clément Laroche, Tobias Piechowiak, Martin Schoeberl, Luca Pezzarossa, Ouassim Karrakchou, Jens Sparsø, Mounir Ghogho
Deep Learning Deep Architecture Early eXit Deep Noise Suppression Early Exiting

August 29, 2023

SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation
Changze Lv, Tianlong Li, Jianhan Xu, Chenxi Gu, Zixuan Ling, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang
Knowledge Distillation Ticket BERT Transformer Based Network Deep Architecture Spiking Transformer Two Stage Knowledge Distillation Spikformer Architecture

August 24, 2023

Master-slave Deep Architecture for Top-K Multi-armed Bandits with Non-linear Bandit Feedback and Diversity Constraints
Hanchi Huang, Li Shen, Deheng Ye, Wei Liu
Multi Armed Bandit Bandit Feedback Deep Architecture Armed Bandit Combinatorial Bandit

August 22, 2023

Using Early Exits for Fast Inference in Automatic Modulation Classification
Elsayed Mohammed, Omar Mashaal, Hatem Abou-Zeid
Deep Learning Deep Neural Network Fast Inference Deep Architecture Early eXit Synthetic Signal Automatic Modulation Classification

July 28, 2023

PatchMixer: Rethinking network design to boost generalization for 3D point cloud understanding
Davide Boscaini, Fabio Poiesi
Strong Generalization 3D Point Cloud Deep Learning Architecture Deep Architecture Point Cloud Understanding Partial Point Cloud Network Design

June 15, 2023

[Experiments & Analysis] Evaluating the Feasibility of Sampling-Based Techniques for Training Multilayer Perceptrons
Sana Ebrahimi, Rishi Advani, Abolfazl Asudeh
Deep Neural Network General Analysis Multi Layer Optical Experiment Task Feasibility Matrix Multiplication Deep Architecture Sampling Based

June 2, 2023

Centered Self-Attention Layers
Ameen Ali, Tomer Galanti, Lior Wolf
Vision Transformer Deep Learning Architecture Self Attention Layer Deep Architecture Similar Representation

Deep Architecture

Papers

Learning to rumble: Automated elephant call classification, detection and endpointing using deep architectures

From Logits to Hierarchies: Hierarchical Clustering made Simple

Computer Vision Model Compression Techniques for Embedded Systems: A Survey

Inversion-DeepONet: A Novel DeepONet-Based Network with Encoder-Decoder for Full Waveform Inversion

Transformers are Universal In-context Learners

Just How Flexible are Neural Networks in Practice?

DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs

Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers

Exploiting the capacity of deep networks only at training stage for nonlinear black-box system identification

Training Neural Networks with Internal State, Unconstrained Connectivity, and Discrete Activations

Automatic Aorta Segmentation with Heavily Augmented, High-Resolution 3-D ResUNet: Contribution to the SEG.A Challenge

Hand Gesture Recognition with Two Stage Approach Using Transfer Learning and Deep Ensemble Learning

Towards a universal mechanism for successful deep learning

Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting

SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation

Master-slave Deep Architecture for Top-K Multi-armed Bandits with Non-linear Bandit Feedback and Diversity Constraints

Using Early Exits for Fast Inference in Automatic Modulation Classification

PatchMixer: Rethinking network design to boost generalization for 3D point cloud understanding

[Experiments & Analysis] Evaluating the Feasibility of Sampling-Based Techniques for Training Multilayer Perceptrons

Centered Self-Attention Layers