Deep Architecture

Deep architectures, encompassing deep neural networks with numerous layers, aim to improve the accuracy and efficiency of machine learning models across diverse applications. Current research focuses on optimizing existing architectures like convolutional neural networks (CNNs) and transformers, exploring techniques such as model compression, early exiting, and novel training strategies to enhance performance and address limitations in resource-constrained environments. These advancements are significant for improving the efficiency and applicability of deep learning in areas like computer vision, natural language processing, and system identification, impacting both scientific understanding and practical deployment of AI systems.

41papers

Papers

May 4, 2025

February 4, 2025

Orientation-aware interaction-based deep material network in polycrystalline materials modeling
Deep Architecture Microstructure Model Orientation Feature Material Segmentation

January 22, 2025

Advanced deep architecture pruning using single filter performance
Convolutional Layer Deep Architecture Deep Learning

December 5, 2024

Graph Neural Networks Need Cluster-Normalize-Activate Modules
Deep Architecture Node Classification Graph Neural Network Dynamic ModulE

November 19, 2024

Data-to-Model Distillation: Data-Efficient Learning Framework
Data Efficient Pre Trained Generative Model Dataset Distillation Deep Architecture

October 15, 2024

Learning to rumble: Automated elephant call classification, detection and endpointing using deep architectures
Audio Spectrogram Transformer Data Detection Meta Classifier Deep Architecture Speech Detection Classification Code

October 10, 2024

From Logits to Hierarchies: Hierarchical Clustering made Simple
Multi Stage Clustering Part Whole Hierarchy Deep Architecture Hierarchical Clustering Pre Trained Second Ranked Logits

August 15, 2024

August 2, 2024

Transformers are Universal In-context Learners
Context Learning Deep Architecture Transformer Megatron Decepticons Deep Transformer Vision Transformer

June 17, 2024

Just How Flexible are Neural Networks in Practice?
Stochastic Learning Convolutional Neural Network Deep Architecture Full Batch Gradient Descent Practice Mode Neural Network

June 6, 2024

DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs
Visual Token Deep Architecture Large Multimodal Model Large Language Model Deep Framework

June 5, 2024

Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers
Deep Linear Deep Learning Structured Output Feature Learning Convolutional Layer Deep Architecture

December 26, 2023

Exploiting the capacity of deep networks only at training stage for nonlinear black-box system identification
Training Stage Capacity Loss Deep Network Deep Generative Model Deep Architecture Deep Model System Identification Basis Function

December 22, 2023

Training Neural Networks with Internal State, Unconstrained Connectivity, and Discrete Activations
Connectivity Constraint Deep Architecture Neural Network Binary Activation Discrete Activation Training Algorithm Internal State

October 24, 2023

Automatic Aorta Segmentation with Heavily Augmented, High-Resolution 3-D ResUNet: Contribution to the SEG.A Challenge
High Resolution Aorta Segmentation Encoder Decoder Deep Architecture Self Augmentation U Net Client Contribution Quantitative Segmentation

September 20, 2023

Hand Gesture Recognition with Two Stage Approach Using Transfer Learning and Deep Ensemble Learning
Hand Gesture Recognition Convolutional Neural Network Gesture Class Stage Approach Deep Ensemble Deep Architecture Transfer Learning

September 14, 2023

Towards a universal mechanism for successful deep learning
General Purpose Model ImageNet Dataset Deep Learning Signal to Noise Ratio Deep Architecture

August 31, 2023

Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting
Deep Learning Early Exiting Deep Noise Suppression Deep Architecture Early eXit

Deep Architecture

Papers

Wide & Deep Learning for Node Classification

Always Skip Attention

Orientation-aware interaction-based deep material network in polycrystalline materials modeling

Advanced deep architecture pruning using single filter performance

Graph Neural Networks Need Cluster-Normalize-Activate Modules

Data-to-Model Distillation: Data-Efficient Learning Framework

Learning to rumble: Automated elephant call classification, detection and endpointing using deep architectures

From Logits to Hierarchies: Hierarchical Clustering made Simple

Computer Vision Model Compression Techniques for Embedded Systems: A Survey

Inversion-DeepONet: A Novel DeepONet-Based Network with Encoder-Decoder for Full Waveform Inversion

Transformers are Universal In-context Learners

Just How Flexible are Neural Networks in Practice?

DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs

Feature learning in finite-width Bayesian deep linear networks with multiple outputs and convolutional layers

Exploiting the capacity of deep networks only at training stage for nonlinear black-box system identification

Training Neural Networks with Internal State, Unconstrained Connectivity, and Discrete Activations

Automatic Aorta Segmentation with Heavily Augmented, High-Resolution 3-D ResUNet: Contribution to the SEG.A Challenge

Hand Gesture Recognition with Two Stage Approach Using Transfer Learning and Deep Ensemble Learning

Towards a universal mechanism for successful deep learning

Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting