Deep Neural Network

Deep neural networks (DNNs) are complex computational models aiming to mimic the human brain's learning capabilities, primarily focusing on achieving high accuracy and efficiency in various tasks. Current research emphasizes understanding DNN training dynamics, including phenomena like neural collapse and the impact of architectural choices (e.g., convolutional, transformer, and operator networks) and training strategies (e.g., weight decay, knowledge distillation, active learning). This understanding is crucial for improving DNN performance, robustness (including against adversarial attacks and noisy data), and resource efficiency in diverse applications ranging from image recognition and natural language processing to scientific modeling and edge computing.

Papers

January 8, 2025

Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks
Felix Jimenez, Matthias Katzfuss
Deep Neural Network Uncertainty Quantification Epistemic Uncertainty Neural Collapse Skip Connection Deterministic Uncertainty

January 7, 2025

Fixed Points of Deep Neural Networks: Emergence, Stability, and Applications
L. Berlyand, V. Slavin
Deep Neural Network Financial Application Unsupervised Learning Core Stability Path Breaking Emergence DNN Framework Fixed Point Jacobian Matrix

January 6, 2025

January 5, 2025

December 31, 2024

December 28, 2024

December 27, 2024

December 25, 2024

Torque-Aware Momentum
Pranshu Malviya, Goncalo Mordido, Aristide Baratin, Reza Babanezhad Harikandeh, Gintare Karolina Dziugaite, Razvan Pascanu, Sarath Chandar
Deep Neural Network Momentum Based Cross Sectional Momentum

December 24, 2024

Deep Neural Network

Papers

Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks

Fixed Points of Deep Neural Networks: Emergence, Stability, and Applications

Deep-Relative-Trust-Based Diffusion for Decentralized Deep Learning

A Novel Structure-Agnostic Multi-Objective Approach for Weight-Sharing Compression in Deep Neural Networks

PTEENet: Post-Trained Early-Exit Neural Networks Augmentation for Inference Cost Optimization

An Analysis Framework for Understanding Deep Neural Networks Based on Network Dynamics

TreeLUT: An Efficient Alternative to Deep Neural Networks for Inference Acceleration Using Gradient Boosted Decision Trees

Stealthy Backdoor Attack to Real-world Models in Android Apps

Pruning-based Data Selection and Network Fusion for Efficient Deep Learning

Applying Graph Explanation to Operator Fusion

Adaptive Tabu Dropout for Regularization of Deep Neural Network

TeLU Activation Function for Fast and Stable Deep Learning

MAFT: Efficient Model-Agnostic Fairness Testing for Deep Neural Networks via Zero-Order Gradient Search

Delayed Random Partial Gradient Averaging for Federated Learning

Enhancing Adversarial Robustness of Deep Neural Networks Through Supervised Contrastive Learning

Introduction to Graph Neural Networks: A Starting Point for Machine Learning Engineers

Torque-Aware Momentum

Predator Prey Scavenger Model using Holling's Functional Response of Type III and Physics-Informed Deep Neural Networks

Explaining Speaker and Spoof Embeddings via Probing

Understanding Artificial Neural Network's Behavior from Neuron Activation Perspective