Neural Network Layer

Neural network layers are the fundamental building blocks of deep learning models, responsible for transforming data representations and extracting increasingly complex features. Current research focuses on improving layer efficiency (e.g., through compression, approximation, and optimized architectures like transformers), enhancing their interpretability (e.g., via activation pattern analysis), and addressing security vulnerabilities (e.g., backdoor attacks). These advancements are crucial for building more efficient, robust, and trustworthy deep learning systems across diverse applications, ranging from image recognition and natural language processing to scientific simulations and material design.

Papers

October 21, 2024

Identifying Sub-networks in Neural Networks via Functionally Similar Representations
Tian Gao, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Dennis Wei
Neural Network Meaningful Representation Neural Network Model Mechanistic Interpretability Intermediate Representation Neural Network Layer

October 5, 2024

Accelerating Diffusion Transformers with Token-wise Feature Caching
Chang Zou, Xuyang Liu, Ting Liu, Siteng Huang, Linfeng Zhang
Video Generation Diffusion Transformer Video Synthesis Neural Network Layer Feature Model

September 2, 2024

Representing Neural Network Layers as Linear Operations via Koopman Operator Theory
Nishant Suresh Aswani, Saif Eddin Jabari, Muhammad Shafique
Dynamical System Koopman Operator Dynamic Mode Decomposition Nonlinear Activation Neural Network Layer Simple Neural Network

August 16, 2024

NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance
Raphael T. Husistein, Markus Reiher, Marco Eckhoff
Neural Network Machine Learning Neural Architecture Search Training Free Neural Network Layer Performant Neural Network

July 30, 2024

DeepBaR: Fault Backdoor Attack on Deep Neural Network Layers
C. A. Martínez-Mejía, J. Solano, J. Breier, D. Bucko, X. Hou
Adversarial Sample CNN Architecture Neural Network Layer Backdoor Injection Fault Injection Attack Deep Attack

January 25, 2024

Manifold GCN: Diffusion-based Convolutional Neural Network for Manifold-valued Graphs
Martin Hanik, Gabriele Steidl, Christoph von Tycowicz
Convolutional Neural Network Multi Layer Riemannian Geometry Graph Diffusion Latent Manifold Neural Network Layer Manifold Graph

December 13, 2023

Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Jihao Xin, Ivan Ilin, Shunkang Zhang, Marco Canini, Peter Richtárik
Deep Learning Neural Network Layer Compression Ratio Deep Learning Based Bandwidth

December 11, 2023

DYAD: A Descriptive Yet Abjuring Density efficient approximation to linear neural network layers
Sarin Chandy, Varun Gangal, Yi Yang, Gabriel Maggiotti
System Performance Pytorch Model Sparse Matrix Efficient Approximation Dyadic Interaction Neural Network Layer

December 8, 2023

Pruning Convolutional Filters via Reinforcement Learning with Entropy Minimization
Bogdan Musat, Razvan Andonie
Reinforcement Learning Neural Network Neural Network Pruning Structural Pruning Neural Network Layer Entropy Minimization Deep Convolutional Activation Feature Information Theoretic Reward

September 9, 2023

HAct: Out-of-Distribution Detection with Neural Net Activation Histograms
Sudeepta Mondal, Ganesh Sundaramoorthi
Distribution Detection Image Classification Benchmark Neural Network Layer Hand Crafted Descriptor Neural Network Activation

August 12, 2023

Sensitivity-Aware Mixed-Precision Quantization and Width Optimization of Deep Neural Networks Through Cluster-Based Tree-Structured Parzen Estimation
Seyedarmin Azizi, Mahdi Nazemi, Arash Fayyazi, Massoud Pedram
Deep Neural Network Deep Learning Model Mixed Precision Quantization Neural Network Layer Efficient Deep Learning Model Tree Structured Parzen Estimator Neural Network Design First Open Source Width Benchmark

July 19, 2023

A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints
Andrei V. Konstantinov, Lev V. Utkin
Neural Network Neural Network Output Simple Approach Convex Constraint Neural Network Layer Novel Computational Input Constraint Quadratic Constraint

June 9, 2023

Hidden Classification Layers: Enhancing linear separability between classes in neural networks layers
Andrea Apicella, Francesco Isgrò, Roberto Prevete
Neural Network Deep Learning Deep Network Hidden Layer Linear Separability Visual Based Class Neural Network Layer Classification Layer

December 15, 2022

White-box Inference Attacks against Centralized Machine Learning and Federated Learning
Jingyi Ge
Machine Learning Membership Inference Attack White Box Inference Attack Centralized Machine Learning Neural Network Layer

August 1, 2022

Neural network layers as parametric spans
Mattia G. Bergomi, Pietro Vertechi
Neural Network Multi Layer Back Propagation Neural Network Layer Linear Layer

June 20, 2022

Neural Activation Patterns (NAPs): Visual Explainability of Learned Concepts
Alex Bäuerle, Daniel Jönsson, Timo Ropinski
Visual Explanation Concept Learning Activation Pattern Concept Discovery Activation Space Neural Network Layer Neural Network Activation

May 24, 2022

Efficient anti-symmetrization of a neural network layer by taming the sign problem
Nilin Abrahamsen, Lin Lin
Cultural Sign Neural Network Layer Symmetric Neural Network Learning Probabilistic Symmetrization Antisymmetric Function

March 25, 2022

Neural Network Layers for Prediction of Positive Definite Elastic Stiffness Tensors
Charles F. Jekel, Kenneth E. Swartz, Daniel A. White, Daniel A. Tortorelli, Seth E. Watts
Human Prediction Neural Network Layer Stiffness Matrix

December 13, 2021

A Complete Characterisation of ReLU-Invariant Distributions
Jan Macdonald, Stephan Wäldchen
Dynamical Invariant Probability Distribution Probability Measure Neural Network Layer Complete Characterization Invariant Distribution Practical Deep

December 12, 2021

Communication-Efficient Federated Learning for Neural Machine Translation
Tanya Roosta, Peyman Passban, Ankit Chadha
Machine Translation Neural Machine Translation Neural Machine Translation Model Neural Network Layer

Neural Network Layer

Papers

Identifying Sub-networks in Neural Networks via Functionally Similar Representations

Accelerating Diffusion Transformers with Token-wise Feature Caching

Representing Neural Network Layers as Linear Operations via Koopman Operator Theory

NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance

DeepBaR: Fault Backdoor Attack on Deep Neural Network Layers

Manifold GCN: Diffusion-based Convolutional Neural Network for Manifold-valued Graphs

Kimad: Adaptive Gradient Compression with Bandwidth Awareness

DYAD: A Descriptive Yet Abjuring Density efficient approximation to linear neural network layers

Pruning Convolutional Filters via Reinforcement Learning with Entropy Minimization

HAct: Out-of-Distribution Detection with Neural Net Activation Histograms

Sensitivity-Aware Mixed-Precision Quantization and Width Optimization of Deep Neural Networks Through Cluster-Based Tree-Structured Parzen Estimation

A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints

Hidden Classification Layers: Enhancing linear separability between classes in neural networks layers

White-box Inference Attacks against Centralized Machine Learning and Federated Learning

Neural network layers as parametric spans

Neural Activation Patterns (NAPs): Visual Explainability of Learned Concepts

Efficient anti-symmetrization of a neural network layer by taming the sign problem

Neural Network Layers for Prediction of Positive Definite Elastic Stiffness Tensors

A Complete Characterisation of ReLU-Invariant Distributions

Communication-Efficient Federated Learning for Neural Machine Translation