Neural Network

Neural networks are computational models inspired by the structure and function of the brain, primarily aimed at approximating complex functions and solving diverse problems through learning from data. Current research emphasizes improving efficiency and robustness, exploring novel architectures like sinusoidal neural fields and hybrid models combining neural networks with radial basis functions, as well as developing methods for understanding and manipulating the internal representations learned by these networks, such as through hyper-representations of network weights. These advancements are driving progress in various fields, including computer vision, natural language processing, and scientific modeling, by enabling more accurate, efficient, and interpretable AI systems.

Papers

May 27, 2024

BDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Network
Zongkai Zhang, Zidong Xu, Wenming Yang, Qingmin Liao, Jing-Hao Xue
Neural Network 3D Occupancy Occupancy Network
Reference Neural Operators: Learning the Smooth Dependence of Solutions of PDEs on Geometric Deformations
Ze Cheng, Zhongkai Hao, Xiaoqiang Wang, Jianing Huang, Youjia Wu, Xudan Liu, Yiru Zhao, Songming Liu, Hang Su
Neural Network Neural Operator Solution Path Design Optimization Complex Deformation Linear Dependence Geometry Problem Arbitrary Shape
A Real-Time Voice Activity Detection Based On Lightweight Neural
Jidong Jia, Pei Zhao, Di Wang
Neural Network Real Time Voice Activity Detection Audio Stream

May 26, 2024

Detection of decision-making manipulation in the pairwise comparisons method
Michał Strada, Sebastian Ernst, Jacek Szybowski, Konrad Kułakowski
Neural Network Data Detection Partial Ranking Human Decision Pairwise Comparison Manipulation Detection
Graph neural networks with configuration cross-attention for tensor compilers
Dmitrii Khizbullin, Eduardo Rocha de Andrade, Thanh Hau Nguyen, Matheus Pedroza Ferreira, David R. Pugh
Neural Network Graph Drawing Cross Attention Inference Workload Tensor Compiler Neural Graph
LoQT: Low-Rank Adapters for Quantized Pretraining
Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano, Serge Belongie, Vésteinn Snæbjarnarson
Neural Network Efficient Training Tensor Factorization Gradient Update Low Rank Adapter
On Sequential Loss Approximation for Continual Learning
Menghao Waiyan William Zhu, Ercan Engin Kuruoğlu
Neural Network Continual LEArning Loss Function Continual Learning Class Incremental Learning Memory Consolidation Gradient Consistency
Partial train and isolate, mitigate backdoor attack
Yong Li, Han Gao
Neural Network Training Data Backdoor Attack Model Training Social Isolation Partial Execution
High-Performance Temporal Reversible Spiking Neural Networks with $O(L)$ Training Memory and $O(1)$ Inference Cost
JiaKui Hu, Man Yao, Xuerui Qiu, Yuhong Chou, Yuxuan Cai, Ning Qiao, Yonghong Tian, Bo XU, Guoqi Li
Neural Network Inference Cost Reversible Dynamic Energy Efficient Inference Memory Efficient Training Reversible Architecture
Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks
Leyang Zhang, Yaoyu Zhang, Tao Luo
Neural Network Geometric Analysis Two Layer Neural Network Saddle to Saddle Dynamic

May 25, 2024

LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters
Xinyu Zhou, Boris Knyazev, Alexia Jolicoeur-Martineau, Jie Fu
Neural Network Deep Learning Model Transformer Megatron Decepticons Low Rank Many Parameter Graph Hypernetworks
Front-propagation Algorithm: Explainable AI Technique for Extracting Linear Function Approximations from Neural Networks
Javier Viaña
Neural Network Deep Neural Network Explainability Method Linear Function Approximation Front Propagation

May 24, 2024

A rationale from frequency perspective for grokking in training neural network
Zhangchen Zhou, Yaoyu Zhang, Zhi-Qin John Xu
Neural Network Grokking Phenomenon Rationale Alignment Frequency Perspective Frequency Analysis
Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regime
Alistair Shilton, Sunil Gupta, Santu Rana, Svetha Venkatesh
Neural Network Energy Based Model Kernel Based Parameterized Regime Neural Kernel Representer Theorem

May 23, 2024

Understanding the dynamics of the frequency bias in neural networks
Juan Molina, Mircea Petrache, Francisco Sahli Costabal, Matías Courdurier
Neural Network Neural Tangent Kernel Two Layer Neural Network Traditional Neural Network Frequency Bias
Neural Pfaffians: Solving Many Many-Electron Schrödinger Equations
Nicholas Gao, Stephan Günnemann
Neural Network Neural Wave Function Many Electron Schr\"odinger Equation
Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension
Kedar Karhadkar, Michael Murray, Guido Montúfar
Neural Network Neural Tangent Kernel General Bound Smallest Eigenvalue Spherical Data Arbitrary Dimension Independent Variable Collinearity Hemisphere Neural Network
Graphcode: Learning from multiparameter persistent homology using graph neural networks
Michael Kerber, Florian Russold
Neural Network Graph Drawing Multi Scale Many Parameter Topological Feature Persistent Homology
Minimum number of neurons in fully connected layers of a given neural network (the first approximation)
Oleg I. Berngardt
Neural Network Single Neuron Level Cross Validation Fully Connected Layer Minimum Number NN Hyperparameters Truncated Singular Value Decomposition
Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equations
Chuqi Chen, Yahong Yang, Yang Xiang, Wenrui Hao
Neural Network Differential Equation Two Layer Neural Network Automatic Differentiation Shannon Entropy Finite Difference

Neural Network

Papers

BDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Network

Reference Neural Operators: Learning the Smooth Dependence of Solutions of PDEs on Geometric Deformations

A Real-Time Voice Activity Detection Based On Lightweight Neural

Detection of decision-making manipulation in the pairwise comparisons method

Graph neural networks with configuration cross-attention for tensor compilers

LoQT: Low-Rank Adapters for Quantized Pretraining

On Sequential Loss Approximation for Continual Learning

Partial train and isolate, mitigate backdoor attack

High-Performance Temporal Reversible Spiking Neural Networks with $O(L)$ Training Memory and $O(1)$ Inference Cost

Geometry of Critical Sets and Existence of Saddle Branches for Two-layer Neural Networks

LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters

Front-propagation Algorithm: Explainable AI Technique for Extracting Linear Function Approximations from Neural Networks

A rationale from frequency perspective for grokking in training neural network

Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regime

Understanding the dynamics of the frequency bias in neural networks

Neural Pfaffians: Solving Many Many-Electron Schrödinger Equations

Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension

Graphcode: Learning from multiparameter persistent homology using graph neural networks

Minimum number of neurons in fully connected layers of a given neural network (the first approximation)

Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equations