Neural Tangent Kernel

Also known as NTK

Ask about this topic

Overview

The neural tangent kernel (NTK) measures how a network’s outputs change with its parameters, letting researchers approximate gradient-based training as kernel regression, especially in the infinite-width or lazy-training regime. Work studies how kernel spectra and their alignment with labels affect convergence, generalization, and memorization, using tools such as eigenvalue analysis, random matrix theory, and finite-width empirical kernels. Extensions apply these ideas to physics-informed neural networks, attention and recurrent models, uncertainty estimation, and active learning, while randomized trace methods and preconditioning make kernel diagnostics more practical. A central tension is that fixed-kernel theory is tractable and often predictive, but real networks can learn changing features, and the width needed for kernel-limit predictions may be impractical or fail to capture advantages of feature learning.

Latest papers

CardsList

44 papers6 papers in the last four weeks, up 20% on the four weeks before. 0.0% of all new papers.

  1. The proton structure one mode at a time: a Neural Tangent Kernel analysis of a DIS fit

    Oct 26, 2026Amedeo Chiefa, Luigi Del Debbio, Richard KenwayNeural Network Training DynamicsNeural Tangent Kernel

    The Higgs Centre for Theoretical Physics, School of Physics and Astronomy, The University of Edinburgh, Peter Guthrie Tait Road, Edinburgh EH9 3FD, United Kingdom

  2. Active Learning of Conditional Generative Models via the Transport Neural Tangent Kernel

    Oct 26, 2026Jayoung Ryu, Kyunghyun Cho, Romain LopezActive LearningNeural Tangent Kernel

    Department of Computer Science, Courant Institute School of Mathematics, Computing, and Data Science, New York University · Department of Biology, New York University

  3. The Symmetry of Differential Neural Tangent Kernel under the Orthogonal Group

    Oct 25, 2026Bangti Jin, Longjun WuNeural Tangent KernelNeural Network Training Dynamics

    Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, P.R. China

  4. Learning the Geometry of Materials Space with Trained Neural Tangent Kernels

    Oct 20, 2026Daniel Kaplan, Gabriel Kotliar, Anirvan M SenguptaNeural Representation GeometryComputational Materials Science

    Center for Materials Theory, Department of Physics and Astronomy, Rutgers University, Piscataway 08854 New Jersey USA · Department of Physics and Astronomy, Rutgers University, Piscataway 08854 New Jersey USA · Department of Physics, University of Central Florida, Orlando, Florida 32816, USA +3

  5. Global Convergence in Deep Networks via the Second Law of Thermodynamics

    Oct 17, 2026Matan Tsipory, Ofir Gaash, Itamar Harel +2Neural Network Training DynamicsStochastic Optimization Convergence

    Technion - Israel Institute of Technology, The Andrew and Erna Viterbi Faculty of Electrical & Computer Engineering, Technion City, Haifa 3200003, Israel. · Technion - Israel Institute of Technology, Faculty of Data and Decision Sciences, Technion City, Haifa 3200003, Israel. · Toyota Technological Institute at Chicago, Chicago, IL, USA.

  6. Checkable NTK Positivity and Finite-Width Gradient Descent for Scalar- and Vector-Valued PINNs with Strong-Form, Weak-Form, and Nonlocal Linear Constraints

    Oct 3, 2026Zifan LyuPhysics-Informed MLNeural Tangent Kernel

    ETH Zürich

  7. Certified Approximation for Interpretable Representer Landmarks

    Sep 30, 2026Jayanta Mukherjee, Shourya Verma, Mengbo Wang +2Neural Network InterpretabilityKernel Methods

    Department of Computer Science Purdue University West Lafayette, IN 47907, USA

  8. First Learn, Then Memorize: The Spectral Bias of Diffusion Models

    Sep 28, 2026Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1Neural Network GeneralizationMemorization in Generative Models

    Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Department of Computing Sciences, Bocconi University, Milano, Italy

  9. A Spectral Theory of Grokking: Weight Decay induces Feature Learning

    Sep 22, 2026Lenz Pracher, Pascal de Jong, Oskar Lieshaus +2Neural Network GeneralizationWeight Decay

    Arnold-Sommerfeld-Center for Theoretical Physics, Ludwig-Maximilians-Universität München, Munich, Germany · Department of Applied Physics, Standford University, Standford, CA, USA · Department of Mathematics, University of Cambridge, Cambridge, UK

  10. Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels

    Sep 12, 2026Lei Shi, Jia-Qi Yang, Ding-Xuan ZhouOperator LearningKernel Methods

    School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China. · School of Mathematics and Statistics, the University of Sydney, Sydney, New South Wales 2006, Australia.

  11. Latent-MoE: Domain-Aware Mixture-of-Experts for PDEs with Multi-Regime Physics

    Sep 7, 2026Hanwen Wang, Paris PerdikarisNeural PDE SolversPhysics-Informed ML

    Graduate Group of Applied Mathematics and Computational Science University of Pennsylvania Philadelphia, PA 19104 · Department of Mechanical Engineering and Applied Mechanics University of Pennsylvania Philadelphia, PA 19104

  12. Kernel Reboot: Breaking the Boundaries of Neural Tangent Kernels for Neural Fields

    Sep 2, 2026Amir Mallak, Alaa Maalouf, Lior Wolf +2Implicit Neural RepresentationsMeta-Learning

    Department of Computer Science, University of Haifa · Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology · School of Computer Science and AI, Tel Aviv University

  13. AdaptNTK: Adaptive Uncertainty Quantification and Active Learning for Neural Network Potentials

    Aug 31, 2026Prajwal Ananth, Shuwen YueUncertainty QuantificationMachine Learning Interatomic Potentials

    Center for Applied Mathematics, Cornell University, Ithaca, NY, USA · R. F. Smith School of Chemical and Biomolecular Engineering, Cornell University, Ithaca, NY, USA

  14. Empirical Characterization of Learning Geometry in Hybrid Quantum Forecasting Models

    Aug 19, 2026Sandra Leticia Juárez-Osorio, Jorge I. Hernandez-Martinez, Jesus Ivan Ruiz-Martinez +2Quantum Machine LearningHybrid Quantum-Classical ML

    Department of Computer Science, CINVESTAV Guadalajara, Mexico · Cinvestav Unidad Tamaulipas, Mexico

  15. Correlation flow governs learning at criticality

    Aug 8, 2026Andrea Combette, Nelly Pustelnik, Antoine VenailleNeural Network InitializationNeural Network Training Dynamics

    CNRS, ENS de Lyon, Laboratoire de Physique UMR5672, Lyon, France

  16. Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices

    Aug 4, 2026Zhao SongReLU Neural NetworksSpectral Methods

  17. The Differential Neural Tangent Kernel and Its Positivity

    Jul 11, 2026Bangti Jin, Longjun WuNeural PDE SolversPhysics-Informed ML

    Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, P.R. China

  18. A Function-Space Dichotomy for Compositional Learning: Exponential Sub-Optimality of the Neural Tangent Kernel

    Jul 7, 2026Arkaprabha Ganguli, Emil ConstantinescuNeural Network GeneralizationKernel Regression

    Mathematics & Computer Science Division, Argonne National Laboratory, Lemont, IL, USA

  19. Scalable Uncertainty Quantification for Extreme Weather Forecasting via Empirical Neural Tangent Kernels

    Jun 1, 2026Jose Marie Antonio Miñoza, Rex Gregor Laylo, Sebastian C. IbañezUncertainty QuantificationIndependent Component Analysis

    Center for AI Research · Department of Education · Makati, Philippines

  20. Spectral Reach: Understanding Neural Scaling as Progress into the Spectral Tail

    May 29, 2026Konstantin Nikolaou, Jonas Scheunemann, Sven Krippendorf +2Representation LearningNeural Network Training Dynamics

    Institute for Computational Physics, University of Stuttgart, Stuttgart, Germany · Department of Applied Mathematics and Theoretical Physics and Department of Physics, University of Cambridge, Cambridge, UK

  21. Label-NTK Alignments and A Tighter Convergence Bound in the NTK Regime

    May 24, 2026Ruchirinkil Marreddy, Chaoyue LiuNeural Network GeneralizationNeural Tangent Kernel

    Elmore Family School of Electrical and Computer Engineering Purdue University

  22. Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization

    May 22, 2026Youngjae Park, Jaemin Kim, Junghwa HongGauss-Newton OptimizationNeural Network Optimization

    Dept. of Control and Instrumentation Engineering, Korea University, Sejong, South Korea · BK21 FOUR Smart Mobility Education and Research Team, Korea University, Sejong, South Korea

  23. Training Infinitely Deep and Wide Transformers

    May 17, 2026Raphaël Barboni, Maarten V. de Hoop, Takashi Furuya +1TransformerWasserstein Gradient Flows

    Bocconi University · Rice University · Doshisha University, RIKEN AIP +1

  24. The Neural Tangent Kernel for Classification

    May 17, 2026Jonathan Plenk, Sergio Calvo-Ordonez, Alvaro Cartea +3ClassificationNeural Network Training Dynamics

    Mathematical Institute, University of Oxford · Oxford-Man Institute of Quantitative Finance, University of Oxford · OATML, University of Oxford +2

  25. Deciphering Neural Reparameterized Full-Waveform Inversion with Neural Sensitivity Kernel and Wave Tangent Kernel

    May 14, 2026Ruihua Chen, Yisi Luo, Bangyu Wu +2Seismic ImagingNeural Tangent Kernel

    School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, China · School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu 610000, China

  26. Force-Aware Neural Tangent Kernels for Scalable and Robust Active Learning of MLIPs

    May 13, 2026Eszter Varga-Umbrich, Zachary Weller-Davies, Paul Duckworth +3Machine Learning Interatomic PotentialsDistribution Shift

    InstaDeep, UK · Department of Engineering, University of Cambridge, Cambridge, UK

  27. State-Space NTK Collapse Near Bifurcations

    May 12, 2026James Hazelden, Eric Shea-BrownRecurrent Neural NetworksNeural Network Training Dynamics

    University of Washington

  28. The Global Empirical NTK: Self-Referential Bias and Dimensionality of Gradient Descent Learning

    May 9, 2026James Hazelden, Laura Driscoll, Eli Shlizerman +1Representation LearningNeural Tangent Kernel

    Applied Mathematics, University of Washington · Allen Institute · Neurobiology & Biophysics, University of Washington +1

  29. Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit

    May 8, 2026Konstantin Riedl, Konstantinos Spiliopoulos, Justin SirignanoOverparameterized Neural NetworkNeural Network Optimization

    Mathematical Institute University of Oxford · Department of Mathematics & Statistics Boston University

  30. Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer

    May 8, 2026Clarissa Lauditi, Cengiz Pehlevan, Blake BordelonDeep Linear NetworksNeural Network Training Dynamics

    John A. Paulson School of Engineering and Applied Sciences, Harvard University · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University · Center for Mathematical Sciences and Applications, Harvard University +1