Neural Network

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

28 new papers

A weekly snapshot of new work published in Neural Network.

Period ending 2026-09-14

26 new papers

A weekly snapshot of new work published in Neural Network.

Period ending 2026-09-07

27 new papers

A weekly snapshot of new work published in Neural Network.

Inside this field

Focused directions

Focused topic

Neural Network

Within Neural Network

658 papersExplore

Focused topic

Neural Architectures

Within Neural Network

123 papersExplore

Focused topic

Activation Functions

Within Neural Network

109 papersExplore

Focused topic

Deep Network

Within Neural Network

71 papersExplore

Focused topic

Two-Layer Neural Networks

Within Neural Network

63 papersExplore

Focused topic

Rectified Linear Unit Networks

Within Neural Network

49 papersExplore

Focused topic

Neural Tangent Kernel

Within Neural Network

47 papersExplore

Focused topic

Modern Deep Networks

Within Neural Network

44 papersExplore

Focused topic

Rectified Linear Unit

Within Neural Network

30 papersExplore

1,083 papers

Latest in Neural Network

Browse all topics →

Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity
May 9, 2026Zhongjie Shi, Wenjing LiaoTransformer Architectures Approximation
A Reconfigurable Multiplier Architecture for Error-Resilient Applications in RISC-V Core
May 9, 2026Pragun Jaswal, L. Hemanth Krishna, B. SrinivasuRisc-V Sparse Deep Neural Network Accelerators
The Global Empirical NTK: Self-Referential Bias and Dimensionality of Gradient Descent Learning
May 9, 2026James Hazelden, Laura Driscoll, Eli Shlizerman +1Neural Tangent Kernel Recurrent Model
Supersampling Stable Diffusion and Beyond: A Seamless, Training-Free Approach for Scaling Neural Networks Using Common Interpolation Methods
May 9, 2026Md Abu Obaida Zishan, Jannatun Noor, Annajiat Alim RaselLarge-Scale Image Generation Denoising Diffusion Probabilistic Model
Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression
May 8, 2026Mingsong Yan, Dongyang Li, Charles Kulick +1Kernel Ridge Regression Transformer Attention
The Geometric Structure of Models Learning Sparse Data
May 8, 2026Thomas Walker, T. Mitchell Roddenberry, Ahmed Imtiaz Humayun +2Feature Learning Deep Network
Vertex-Softmax: Tight Transformer Verification via Exact Softmax Optimization
May 8, 2026Navid Rezazadeh, Arash Gholami DavoodiGumbel-Softmax Relaxation Transformer Attention
Queryable LoRA: Instruction-Regularized Routing Over Shared Low-Rank Update Atoms
May 8, 2026Omatharv Bharat Vaidya, Connor T. Jerzak, Nhat Ho +1Low-Rank Adaptation Parameter-Efficient Fine-Tuning
Transfer Learning for Dead Fuel Moisture Prediction Using Time-Warping Recurrent Neural Networks
May 8, 2026Jonathon Hirschi, Jan Mandel, Adam KochanskiTransfer Learning Neural Network
Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit
May 8, 2026Konstantin Riedl, Konstantinos Spiliopoulos, Justin SirignanoNeural Tangent Kernel Over-Parameterized Neural Networks
6D Pose Estimation via Keypoint Heatmap Regression with RGB-D Residual Neural Networks
May 8, 2026Ismail Aljosevic, Amir Masoud Almasi, Ana Parovic +1Category-Level Object Pose Estimation Keypoint Detection
FQPDR: Federated Quantum Neural Network for Privacy-preserving Early Detection of Diabetic Retinopathy
May 8, 2026Debashis De, Mahua Nandy Pal, Dipankar HazraQuantum Neural Networks Mnist
Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation
May 8, 2026Joon Ha Kim, Geon-Woo Kim, Anoop Rachakonda +1Profiling LLM Inference Efficiency
Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions
May 8, 2026Marwane Hariat, Antoine Manzanera, David FilliatDepth Estimation Optical Flow
Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer
May 8, 2026Clarissa Lauditi, Cengiz Pehlevan, Blake BordelonDeep Network Neural Network
When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains
May 8, 2026Brandon Yee, Pairie Koh, Jack Rodriguez +1Partial Differential Equation Benchmarks Fourier
Efficient Verification of Neural Control Barrier Functions with Smooth Nonlinear Activations
May 8, 2026Jun Zhang, Haibo Zhang, Chun Liu +2Control Barrier Functions Neural Control
VNN-LIB 2.0: Rigorous Foundations for Neural Network Verification
May 8, 2026Ann Roy, Allen Antony, Andrea Gimelli +1Neural Network Verification Formal Verification
Sparse Random-Feature Neural Networks with Krylov-Based SVD for Singularly Perturbed ODE
May 8, 2026Kevin Kurian Thomas Vaidyan, Siddharth RoutSingular Value Decomposition Radial Basis Function
Solving Max-Cut to Global Optimality via Feasibility-Preserving Graph Neural Networks
May 8, 2026Hao Chen, Chendi Qian, Christopher Morris +2Semidefinite Programming Maximum Independent Set
Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity
May 8, 2026Anastasis Kratsios, Gregory Cousins, Haitz Sáez de Ocáriz Borde +2Feed-Forward Network Multilayer Perceptrons
Causal EpiNets: Precision-corrected Bounds on Individual Treatment Effects using Epistemic Neural Networks
May 8, 2026Gandharv Patil, Keyi Tang, Raquel Aoki +1Heterogeneous Treatment Effects Finite-Sample
Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks
May 8, 2026Ryoichiro Agata, Tomohisa OkazakiConventional Physics-Informed Neural Networks Bayesian
A Generalized Singular Value Theory for Neural Networks
May 7, 2026Brian Charles Brown, Robert Bridges, David Grimsman +2Singular Value Decomposition Equivariant Neural Networks
CARMEN: CORDIC-Accelerated Resource-Efficient Multi-Precision Inference Engine for Deep Learning
May 7, 2026Sonu Kumar, Mukul Lokhande, Santosh Kumar Vishvakarma +1Sparse Deep Neural Network Accelerators Real-Time Inference
Attribution-Based Neuron Utility for Plasticity Restoration in Deep Networks
May 7, 2026Patrick Elisii, Lucas Beauchemin, Dawer JamshedEfficient Continual Learning Plasticity
Criticality and Saturation in Orthogonal Neural Networks
May 7, 2026Max Guillen, Jan E. GerkenTwo-Layer Neural Networks Orthogonality
Lecture Notes on Statistical Physics and Neural Networks
May 7, 2026Olaf HohmStatistical Mechanics Two-Dimensional Ising Model
Region Seeding via Pre-Activation Regularization: A Geometric View of Piecewise Affine Neural Networks
May 7, 2026Yi Wei, Xuan Qi, Furao ShenAffine Deep Network
ConquerNet: Convolution-Smoothed Quantile ReLU Neural Networks with Minimax Guarantees
May 7, 2026Tianpai Luo, Fangwei Wu, Weichi WuQuantile Regression Rectified Linear Unit Networks
The Weight Gram Matrix Captures Sequential Feature Linearization in Deep Networks
May 7, 2026Taehun Cha, Daniel Beaglehole, Adityanarayanan Radhakrishnan +1Feature Learning Deep Network
VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
May 7, 2026Keisuke Kamahori, Shihang Li, Simon Peter +1Large Language Model Serving Large Language Model Agents
Normalized Architectures are Natively 4-Bit
May 7, 2026Maxim Fishman, Brian Chmiel, Ron Banner +2Full-Precision Performance Low-Bit
Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networks
May 7, 2026Ying Chen, Aoxi Li, Jihun Kim +1Low-Rank Factorization Neural Architectures
Permutation-preserving Functions and Neural Vecchia Covariance Kernels
May 6, 2026Jian Cao, Nian Liu, Ying LinGaussian Process Covariance Estimation
Adaptivity Under Realizability Constraints: Comparing In-Context and Agentic Learning
May 6, 2026Anastasis Kratsios, A. Martina Neuman, Philipp PetersenAdaptive Learning In-Context Learning
Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
May 6, 2026Xuan Qi, Yi Wei, Fanqi Yu +3Batch Normalization Affine
Quantile-Free Uncertainty Quantification in Graph Neural Networks
May 6, 2026Soyoung park, Hwanjun Song, Sungsu LimReliable Uncertainty Quantification Graph Neural Networks
Benchmarking POS Tagging for the Tajik Language: A Comparative Study of Neural Architectures on the TajPersParallel Corpus
May 6, 2026Mullosharaf K. ArabovNeologisms Multilingual Language Models
FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning
May 6, 2026Waleed Razzaq, Yun-Bo ZhaoTransformer Architectures Efficient Continual Learning
On the Architectural Complexity of Neural Networks
May 5, 2026Nicholas J. Cooper, François G. Meyer, Michael L. Roberts +3Neural Architectures Neural Network
Probabilistic Classification and Uncertainty Quantification of Sahara Desert Climate Using Feedforward Neural Networks
May 5, 2026Stephen Tivenan, Indranil Sahoo, Yanjun QianClimate Neural Network
CP $^2$ N $^2$ : A Control Plane for Heterogeneous Physical Neural Networks
May 5, 2026Stefan Fischer, Maliheh Hariri, Sebastian OtteEdge Computing Neural Network
Layerwise LQR for Geometry-Aware Optimization of Deep Networks
May 5, 2026Simon Dufort-Labbé, Pierre-Luc Bacon, Razvan Pascanu +2Neural Network Optimization Spectral Preconditioning
Exact ReLU realization of tensor-product refinement iterates
May 5, 2026Tsogtgerel GantumurRectified Linear Unit Iterative Refinement
Graph Neural Networks in the Wilson Loop Representation of Abelian Lattice Gauge Theories
May 5, 2026Ali Rayat, Gia-Wei ChernEffective Field Theory Graph Neural Networks
Spatiotemporal Convolutions on EEG signal -- A Representation Learning Perspective on Efficient and Explainable EEG Classification with Convolutional Neural Nets
May 5, 2026Laurits Dixen, Stefan Heinrich, Paolo BurelliTemporal Convolutional Networks Convolutional
Rethinking the Rank Threshold for LoRA Fine-Tuning
May 5, 2026Juneyoung ParkLow-Rank Adaptation Cross-Entropy Losses
Information Plane Analysis of Binary Neural Networks
May 5, 2026Maximilian Nothnagel, Bernhard C. GeigerNeural Network Mutual Information
Most ReLU Networks Admit Identifiable Parameters
May 5, 2026Moritz Grillo, Guido MontúfarRectified Linear Unit Networks Intrinsic Dimensionality
Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks
May 5, 2026Jatin Sharma, Dan F. M Goodman, Danyal AkarcaRecurrent Neural Networks Neural Network
Random test functions, $H^{-1}$ norm equivalence, and stochastic variational physics-informed neural networks
May 5, 2026Diego MarcondesPartial Differential Equation Benchmarks Logarithmic Sobolev Inequalities
Parametrizing Convex Sets Using Sublinear Neural Networks
May 5, 2026Eloi MartinetConvex Sets Neural-Network Approximation
Meta-Inverse Physics-Informed Neural Networks for High-Dimensional Ordinary Differential Equations
May 5, 2026Zhao Wei, Kenneth Hor Cheng Koh, Sheng Yuan Chin +3Conventional Physics-Informed Neural Networks Ordinary Differential Equations
Physics-Modeled Neural Networks
May 5, 2026Raul Felipe-Sosa, Angel Martin del Rey, Maria Flores CeballosNeural Ordinary Differential Equations Neural Network
GRAFT: Auditing Graph Neural Networks via Global Feature Attribution
May 5, 2026Rishi Raj Sahoo, Subhankar MishraGraph Neural Networks Subgraphs
Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective
May 5, 2026Zhe Li, Bicheng Ying, Zidong Liu +1Zeroth-Order Optimization Neural Tangent Kernel
Symmetry, Defects, and Diffusion in Continuous-memory Recurrent Networks
May 5, 2026Hanson Hanxuan MoRecurrent Neural Networks Symmetry
Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count
May 4, 2026Lurong PanTwo-Layer Neural Networks Condition Number
Trust, but Verify: Peeling Low-Bit Transformer Networks for Training Monitoring
May 4, 2026Arian Eamaz, Farhang Yeganegi, Mojtaba SoltanalianTransformer Layers Transformer Architectures

May 9, 2026 · stat.MLJ/K move · Enter open · S save

Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity

Zhongjie Shi, Wenjing Liao

School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332, United States

This paper investigates the learning theory of Transformer networks for regression tasks on the compact Euclidean domain

[0,1]^d

and

d

-dimensional compact Riemannian manifolds. We propose a novel constructive approximation framework for Transformers that builds local approximations of the target function and aggregates them into a global approximation via softmax partition of unity. This approach leverages the attention mechanism to achieve spatial localization through affine transformations of the input. The softmax activation plays a crucial role in aggregating local approximations to a global output. From an approximation perspective, we prove that a dense Transformer equipped with only two encoder blocks and standard single-hidden-layer point-wise feed-forward networks can achieve a uniform

\varepsilon

-approximation error for

α

-Hölder continuous functions with

α\in (0,1]

using

\mathcal{O}(\varepsilon^{-d/α})

total parameters. Building upon this approximation guarantee, we establish a near minimax-optimal generalization error bound of order

\mathcal{O}\big(n^{-\frac{2α}{2α+d}} \log n\big)

for the empirical risk minimizer, where

n

is the training data size. The Transformer architecture studied in this paper is dense, shallow and wide, and employs softmax activation and sinusoidal positional encodings, closely reflecting practical implementations.

Transformer Architectures Approximation Activation Functions Generalization Bounds Feed-Forward Network Partitioning

Open paper Ask about it PDF