Modern Deep Learning Model

Modern deep learning models aim to achieve high accuracy and generalization performance across diverse tasks, but face challenges like catastrophic forgetting and inefficient training on large, varied datasets. Current research focuses on improving model robustness and efficiency through techniques such as knowledge distillation, attention mechanisms inspired by human cognition, and refined optimization strategies like sharpness-aware minimization to find flatter minima in the loss landscape. These advancements are crucial for building more reliable and adaptable AI systems with broader practical applications, particularly in resource-constrained environments.

Papers

October 29, 2024

Effective Guidance for Model Attention with Simple Yes-no Annotations
Seongmin Lee, Ali Payani, Duen Horng (Polo) Chau
Saliency Map Modern Deep Learning Model Attention Based Reasoning

October 11, 2024

On the Adversarial Transferability of Generalized "Skip Connections"
Yisen Wang, Yichuan Mo, Dongxian Wu, Mingjie Li, Xingjun Ma, Zhouchen Lin
Adversarial Example ResNet Model Modern Deep Learning Model

February 28, 2024

Trends, Applications, and Challenges in Human Attention Modelling
Giuseppe Cartella, Marcella Cornia, Vittorio Cuculo, Alessandro D'Amelio, Dario Zanca, Giuseppe Boccignone, Rita Cucchiara
Technical Challenge Financial Application Vision Language Attention Mechanism Recent Trend Attention Model Modern Deep Learning Model

December 16, 2023

Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Everton L. Aleixo, Juan G. Colonna, Marco Cristo, Everlandio Fernandes
Deep Learning Deep Learning Model Gradient Descent Catastrophic Forgetting Comprehensive Taxonomy Learning Algorithm Modern Deep Learning Model

October 28, 2023

On Training Implicit Meta-Learning With Applications to Inductive Weighing in Consistency Regularization
Fady Rezk
Financial Application Consistency Regularization Implicit Neural Network Weight Freezing Hessian Vector Product Order Gradient Modern Deep Learning Model Implicit Gradient

October 17, 2023

Relearning Forgotten Knowledge: on Forgetting, Overfit and Training-Free Ensembles of DNNs
Uri Stern, Daphna Weinshall
Deep Neural Network Anti Forgetting Deep Model DNN Framework Training Free Model Overfitting Modern Deep Learning Model

October 16, 2023

BLoad: Enhancing Neural Network Training with Efficient Sequential Data Handling
Raphael Ruschel, A. S. M. Iftekhar, B. S. Manjunath, Suya You
Neural Network Neural Network Model Scalable Training Modern Deep Learning Model Sequential Computation Training Scheme

October 3, 2023

July 26, 2023

Sparse Double Descent in Vision Transformers: real or phantom threat?
Victor Quétu, Marta Milovanovic, Enzo Tartaglione
Vision Transformer Sparsity Increase Phantom Threat Modern Deep Learning Model Sparse Double Descent

March 7, 2023

CUDA: Convolution-based Unlearnable Datasets
Vinu Sankar Sadasivan, Mahdi Soltanolkotabi, Soheil Feizi
Adversarial Training Modern Deep Learning Model Unlearnable Datasets Convolution Based Unlearnable

March 2, 2023

DSD$^2$: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?
Victor Quétu, Enzo Tartaglione
Strong Generalization Sparsity Increase Neural Solver Modern Deep Learning Model Sparse Double Descent

February 13, 2023

Fixing Overconfidence in Dynamic Neural Networks
Lassi Meronen, Martin Trapp, Andrea Pilzer, Le Yang, Arno Solin
Deep Learning Model Supervised ImageNet Dynamic Neural Network Modern Deep Learning Model

December 7, 2022

Improved Deep Neural Network Generalization Using m-Sharpness-Aware Minimization
Kayhan Behdin, Qingquan Song, Aman Gupta, David Durfee, Ayan Acharya, Sathiya Keerthi, Rahul Mazumder
Adversarial Perturbation Generalization Performance Sharpness Aware Minimization Neural Network Generalization Modern Deep Learning Model