Fully Connected Neural Network

Fully connected neural networks (FCNNs), characterized by their dense connections between layers, are a fundamental architecture in deep learning, primarily used for tasks like classification and regression. Current research focuses on improving FCNN efficiency and understanding their theoretical properties, including investigations into training dynamics, implicit biases, and the relationship between network architecture and generalization ability. This includes exploring variations like Deep Equilibrium models and employing optimization techniques such as incremental optimization and adaptive learning rates to enhance training speed and performance. The insights gained from this research are crucial for advancing both the theoretical understanding of deep learning and the development of more efficient and effective machine learning models across diverse applications.

Papers

June 29, 2023

Sampling weights of deep neural networks
Erik Lien Bolager, Iryna Burak, Chinmay Datar, Qing Sun, Felix Dietrich
Deep Neural Network Deep Network Neural Operator Balancing Weight Shallow Network Fully Connected Neural Network Efficient Sampling

May 26, 2023

Live American Sign Language Letter Classification with Convolutional Neural Networks
Kyle Boone, Ben Wurster, Seth Thao, Yu Hen Hu
Neural Network Convolutional Neural Network Deep Convolutional Neural Network American Sign Language Fully Connected Neural Network

May 22, 2023

Training an Ising Machine with Equilibrium Propagation
Jérémie Laydevant, Danijela Markovic, Julie Grollier
Ising Model Fully Connected Neural Network Ising Machine Equilibrium Propagation Mechanical Engineering

February 21, 2023

SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics
Emmanuel Abbe, Enric Boix-Adsera, Theodor Misiakiewicz
Neural Network Stochastic Gradient Descent Fully Connected Neural Network Gradient Complexity Saddle to Saddle Dynamic Isotropic Gaussian Complexity Scaling

October 25, 2022

Learning Ability of Interpolating Deep Convolutional Neural Networks
Tian-Yi Zhou, Xiaoming Huo
Convolutional Neural Network Deep Neural Network Deep Convolutional Neural Network Fully Connected Neural Network

October 13, 2022

Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
Spencer Frei, Gal Vardi, Peter L. Bartlett, Nathan Srebro, Wei Hu
Gradient Descent Low Rank Gradient Flow High Dimensional Data Implicit Bias Fully Connected Neural Network Homogeneous Neural Network Leaky ReLU

July 5, 2022

Correlation between entropy and generalizability in a neural network
Ge Zhang
Neural Network Stronger Generalizability Total Correlation Complex Deep Fully Connected Neural Network Entropy Solution Wang Landau Algorithm

May 17, 2022

Spiking Neural Operators for Scientific Machine Learning
Adar Kahana, Qian Zhang, Leonard Gleyzer, George Em Karniadakis
Physic Informed Neural Network Scientific Machine Learning Fully Connected Neural Network SNN Based Regression Function

April 1, 2022

Optimising Communication Overhead in Federated Learning Using NSGA-II
José Ángel Morell, Zakaria Abdelmoiz Dahi, Francisco Chicano, Gabriel Luque, Enrique Alba
Evolutionary Computation FedAvg Algorithm Fully Connected Neural Network Non Dominated Sorting Genetic Algorithm Communication Optimization

March 27, 2022

On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks
Hongru Yang, Zhangyang Wang
Neural Network Neural Tangent Kernel Random Participation Fully Connected Neural Network Random Pruning

January 26, 2022

Physics-informed ConvNet: Learning Physical Field from a Shallow Neural Network
Pengpeng Shi, Zhi Zeng, Tianshou Liang
Physic Informed Machine Learning Physic Informed Shallow Neural Network Fully Connected Neural Network Shallow Layer Physical Law Learning Nonlinear Operator Mapping Wave Speed

January 3, 2022

A Mixed-Integer Programming Approach to Training Dense Neural Networks
Vrishabh Patil, Yonatan Mintz
Neural Network Mixed Integer Fully Connected Neural Network Dense Neural Network Parsimonious Machine Learning