Mathematical Foundations of Deep Learning

Xiaojing Ye

cs.LGMar 19, 2026

Mathematical Foundations of Deep Learning

Authors: Xiaojing Ye

Abstract

This draft book offers a comprehensive and rigorous treatment of the mathematical principles underlying modern deep learning. The book spans core theoretical topics, from the approximation capabilities of deep neural networks, the theory and algorithms of optimal control and reinforcement learning integrated with deep learning techniques, to contemporary generative models that drive today's advances in artificial intelligence.

Neural Network Approximation Theory

Neural network approximation theory studies how network size, depth, activations, and structure determine the accuracy of functions and operators.

See all 236 papers

From Approximation to Emergence: A Theory of Deep Learning
Jul 1, 2026Zhilin ZhaoNeural Network Generalization Deep Learning Optimization
Sun Yat-sen University
Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory
Jun 4, 2026Sam Buchanan, Druv Pai, Peng Wang +1Deep Learning Optimization Representation Learning
UC Berkeley · TTIC · University of Macau +2
The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer
May 28, 2026Tianhua ChenVariational Autoencoders Generative Modeling
School of Computing and Engineering[cs.LG] University of Huddersfield

Jul 1, 2026J/K move · Enter open · S save

From Approximation to Emergence: A Theory of Deep Learning

Zhilin Zhao

Sun Yat-sen University

Neural Network Generalization Deep Learning Optimization In-Context Learning Neural Network Approximation Theory Neural Scaling Laws

Open paper AskPDF arXiv

Deep learning has outgrown any single mathematical explanation. From Approximation to Emergence develops a unified, proof-oriented account of modern deep learning theory, tracing a path from the classical foundations of approximation, optimization, and generalization to the contemporary mechanisms of overparameterization, robustness, generative modeling, transformers, in-context learning, scaling laws, interpretability, alignment, and emergence. Rather than presenting isolated results, the book organizes a broad literature into a coherent research narrative: each theory is examined through the object it controls, the assumptions that make it valid, and the phenomena it leaves unexplained. Written for researchers, graduate students, and mathematically trained practitioners, this monograph offers a rigorous map of deep learning theory as it stands today: powerful, incomplete, and increasingly centered on the question of how learned mechanisms arise from scale, data, architecture, and training.

Abstract

Neural Network Approximation Theory

Explore similar work

From Approximation to Emergence: A Theory of Deep Learning

Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory

The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer