Deep Network

Deep networks, complex artificial neural networks with multiple layers, aim to learn intricate patterns from data by approximating complex functions. Current research focuses on improving their efficiency (e.g., through dataset distillation and novel activation functions), enhancing their interpretability (e.g., via re-label distillation and analysis of input space mode connectivity), and addressing challenges like noisy labels and domain shifts. These advancements are crucial for expanding the applicability of deep networks across diverse fields, from financial modeling and medical image analysis to time series classification and natural language processing, while simultaneously increasing their reliability and trustworthiness.

Papers

October 4, 2022

How deep convolutional neural networks lose spatial information with training
Umberto M. Tomasini, Leonardo Petrini, Francesco Cagnetta, Matthieu Wyart
Training Data Deep Convolutional Neural Network Deep Network Neural Representation Spatial Information Model Scale Spatial Pooling Time Continuous Diffeomorphisms

October 3, 2022

Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networks
Xiang Wang, Annie N. Wang, Mo Zhou, Rong Ge
Deep Neural Network Deep Network Interpolation Regime Loss Landscape Best View Linear Interpolation

October 2, 2022

The Dynamic of Consensus in Deep Networks and the Identification of Noisy Labels
Daniel Shwartz, Uri Stern, Daphna Weinshall
Deep Neural Network Deep Network Noisy Label Person Identification Noisy Data Consensus Group Decision Noisy Example

September 27, 2022

Scaling Laws For Deep Learning Based Image Reconstruction
Tobit Klug, Reinhard Heckel
Deep Neural Network Super Resolution Deep Network Image Denoising Scaling Law Power Law Scaling Learning Based Image Reconstruction

September 21, 2022

September 16, 2022

Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks
Litian Liang, Yaosheng Xu, Stephen McAleer, Dailin Hu, Alexander Ihler, Pieter Abbeel, Roy Fox
Deep Network Diverse Ensemble Variance Reduction Temporal Difference Learning Temporal Difference Variance Estimation Atari Benchmark

September 15, 2022

Theoretical Insight into Batch Normalization: Data Dependant Auto-Tuning of Regularization Rate
Lakshmi Annamalai, Chetan Singh Thakur
Deep Network Regularization Model Batch Normalization Theoretical Analysis Regularization Parameter Auto Tuning

September 12, 2022

September 10, 2022

Deep Baseline Network for Time Series Modeling and Anomaly Detection
Cheng Ge, Xi Chen, Ming Wang, Jin Wang
Deep Learning Anomaly Detection Time Series Deep Network Neural Network Baseline

September 7, 2022

A simple approach for quantizing neural networks
Johannes Maly, Rayan Saab
Neural Network Deep Network Multiplier Free Quantization Simple Approach

August 30, 2022

Robustness and invariance properties of image classifiers
Apostolos Modas
Native Robustness Adversarial Example Adversarial Perturbation Deep Network Image Classifier Invariance Property State of the Art Robustness

August 23, 2022

The Value of Out-of-Distribution Data
Ashwin De Silva, Rahul Ramesh, Carey E. Priebe, Pratik Chaudhari, Joshua T. Vogelstein
Deep Network Distribution Data Generalization Error Net Present Value Distribution Sample Provable Generalization

Deep Network

Papers

How deep convolutional neural networks lose spatial information with training

Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networks

The Dynamic of Consensus in Deep Networks and the Identification of Noisy Labels

Scaling Laws For Deep Learning Based Image Reconstruction

Uncertainty-aware Label Distribution Learning for Facial Expression Recognition

Deep Double Descent via Smooth Interpolation

Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection

Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold

Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty

Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks

Theoretical Insight into Batch Normalization: Data Dependant Auto-Tuning of Regularization Rate

The Mori-Zwanzig formulation of deep learning

Boosting Robustness Verification of Semantic Feature Neighborhoods

Deep Baseline Network for Time Series Modeling and Anomaly Detection

A simple approach for quantizing neural networks

Robustness and invariance properties of image classifiers

The Value of Out-of-Distribution Data

Restructurable Activation Networks

An Evolutionary, Gradient-Free, Query-Efficient, Black-Box Algorithm for Generating Adversarial Instances in Deep Networks

How does the degree of novelty impacts semi-supervised representation learning for novel class retrieval?