Deep Network

Deep networks, complex artificial neural networks with multiple layers, aim to learn intricate patterns from data by approximating complex functions. Current research focuses on improving their efficiency (e.g., through dataset distillation and novel activation functions), enhancing their interpretability (e.g., via re-label distillation and analysis of input space mode connectivity), and addressing challenges like noisy labels and domain shifts. These advancements are crucial for expanding the applicability of deep networks across diverse fields, from financial modeling and medical image analysis to time series classification and natural language processing, while simultaneously increasing their reliability and trustworthiness.

Papers

June 25, 2022

Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels
Mohamad Amin Mohamadi, Wonho Bae, Danica J. Sutherland
Active Learning Deep Network Neural Tangent Kernel Sequential Learning Pool Based Active Learning

June 24, 2022

Learning sparse features can lead to overfitting in neural networks
Leonardo Petrini, Francesco Cagnetta, Eric Vanden-Eijnden, Matthieu Wyart
Neural Network Deep Network Neural Representation Feature Learning Sparse Feature Smooth Representation

June 23, 2022

June 18, 2022

Piecewise Linear Neural Networks and Deep Learning
Qinghua Tao, Li Li, Xiaolin Huang, Xiangming Xi, Shuning Wang, Johan A. K. Suykens
Deep Learning Deep Network Piecewise Linear

June 17, 2022

June 13, 2022

Rank Diminishing in Deep Neural Networks
Ruili Feng, Kecheng Zheng, Yukun Huang, Deli Zhao, Michael Jordan, Zheng-Jun Zha
Deep Neural Network Deep Network Stable Rank Human Rating Low Rank Structure

June 12, 2022

A Functional Information Perspective on Model Interpretation
Itai Gat, Nitay Calderon, Roi Reichart, Tamir Hazan
Inherent Interpretability Deep Network Model Interpretability Fisher Information Model Interpretation Differential Entropy

June 7, 2022

Neural Network Compression via Effective Filter Analysis and Hierarchical Pruning
Ziqi Zhou, Li Lian, Yilong Yin, Ze Wang
Deep Network Neural Network Compression Network Pruning Network Compression Hierarchical Pruning

June 6, 2022

June 2, 2022

Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks
Ali Karimi, Zahra Mousavi Kouzehkanan, Reshad Hosseini, Hadi Asheri
Loss Function Deep Network Cross Entropy Loss Classification Problem Margin Classifier Margin Loss

June 1, 2022

Rotate the ReLU to implicitly sparsify deep networks
Nancy Nayak, Sheetal Kalyani
Deep Neural Network Deep Network Deep Architecture

May 30, 2022

FedAUXfdp: Differentially Private One-Shot Federated Distillation
Haley Hoech, Roman Rischke, Karsten Müller, Wojciech Samek
Federated Learning Deep Network Federated Distillation Local Datasets FedCF Method

May 26, 2022

Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks
Zhiwei Bai, Tao Luo, Zhi-Qin John Xu, Yaoyu Zhang
Deep Neural Network Deep Network Large Depth General Principle Loss Landscape Deeper Network

May 24, 2022

May 23, 2022

Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation
Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield
Deep Network Uncertainty Estimation Single RGB Image Video Annotation Category Level 6D Pose

May 22, 2022

Visual Explanations from Deep Networks via Riemann-Stieltjes Integrated Gradient-based Localization
Mirtha Lucas, Miguel Lerma, Jacob Furst, Daniela Raicu
Convolutional Neural Network Deep Network Natural Gradient Visual Explanation Integrated Gradient Grad CAM Riemann Lebesgue

Deep Network

Papers

Making Look-Ahead Active Learning Strategies Feasible with Neural Tangent Kernels

Learning sparse features can lead to overfitting in neural networks

Single-phase deep learning in cortico-cortical networks

Restoring speech intelligibility for hearing aid users with deep learning

Piecewise Linear Neural Networks and Deep Learning

Decentralized adaptive clustering of deep nets is beneficial for client collaboration

How You Start Matters for Generalization

Rank Diminishing in Deep Neural Networks

A Functional Information Perspective on Model Interpretation

Neural Network Compression via Effective Filter Analysis and Hierarchical Pruning

Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime

Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networks

Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks

Rotate the ReLU to implicitly sparsify deep networks

FedAUXfdp: Differentially Private One-Shot Federated Distillation

Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks

Naive Few-Shot Learning: Uncovering the fluid intelligence of machines

Improving Shape Awareness and Interpretability in Deep Networks Using Geometric Moments

Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation

Visual Explanations from Deep Networks via Riemann-Stieltjes Integrated Gradient-based Localization