Automatic Differentiation

Ask about this topic

Overview

Automatic differentiation (AD) computes derivatives by applying the chain rule to the operations in a program, using reverse mode for many-input, few-output problems and forward or Taylor-mode methods for other derivative structures. Research develops higher-order and sparse differentiation, compiler and runtime systems, and implicit or solver-level adjoints that reduce memory and computation in settings such as neural networks, PDE solvers, and differentiable physical simulation. These methods support tasks including training physics-informed neural networks, parameter estimation, and optimization through event-driven or contact-rich systems. A central tension is that exact gradients of a computation can be expensive or may not reflect the desired physical sensitivity, so work compares AD with finite differences, spectral and discretization-based methods, and structure-aware derivative schemes.

Latest papers

CardsList

33 papers5 papers in the last four weeks, down 29% on the four weeks before. 0.0% of all new papers.

  1. Automatic Differentiation with Constraints in Forward and Inverse Problems

    Oct 20, 2026Nikolay Yavich, Alexander Ryabov, Viacheslav Naumov +4PDE SolvingPDE Inverse Problems

    Artificial Intelligence Center, Skolkovo Institute of Science and Technology, 121205 Moscow, Russia · Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi, Hauz Khas, New Delhi, India 110016 · Learnable Intelligence Group, Autonomous Non-Profit Organization Artificial Intelligence Research Institute (AIRI), 105064 Moscow, Russia

  2. Coordinate-Free Automatic Differentiation

    Oct 8, 2026Alex Guerra, Joshua Aduol, Yaqian Tang +3Automatic DifferentiationDifferentiable Programming

    Department of Computer Science, Princeton University, Princeton, NJ 08540 · Gatsby Computational Neuroscience Unit, University College London

  3. asdex: Automatic Sparse Differentiation in JAX

    Oct 8, 2026Adrian Hill, Guillaume DalleAutomatic DifferentiationStructured Sparsity

    BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · Machine Learning Group, Technical University of Berlin, Berlin, Germany · LVMT, ENPC, Institut Polytechnique de Paris, Univ Gustave Eiffel, Marne-la-Vallée, France

  4. Recursive Improvement of a Differentiable Scientific Software Ecosystem

    Oct 3, 2026Pengcheng Hou, Xiaojun Tan, Sihan Hu +5Differentiable ProgrammingAutomatic Differentiation

    Hefei National Laboratory, Hefei 230088, China · Institute for Advanced Algorithms Research, Shanghai 200120, China · University of Science and Technology of China, Hefei, Anhui 230026, China +2

  5. Same Loss, Different Gradients

    Sep 30, 2026Ningkang Peng, Xiaoqian Peng, Yifan He +4Automatic DifferentiationDifferentiable Optimization

    Nanjing Normal University · Nanjing University of Chinese Medicine

  6. The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning

    Sep 27, 2026Ameya D. JagtapAutomatic DifferentiationPDE Surrogate Modeling

    Aerospace Engineering Department, Worcester Polytechnic Institute, Worcester, MA 01609, USA

  7. FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention

    Sep 21, 2026Anthony Givans, Michael Crawshaw, Mingrui LiuSoftmax AttentionAutomatic Differentiation

  8. Truncated automatic sparse differentiation for machine learning interatomic potentials

    Sep 17, 2026Marcel F. Langer, Adrian Hill, Michele CeriottiAutomatic DifferentiationMachine Learning Interatomic Potentials

    Laboratory of Computational Science and Modeling (COSMO), EPFL, Lausanne, Switzerland · Machine Learning Group, Technical University of Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany

  9. Rank and computation of the pathlifting Jacobian of a DAG ReLU network

    Sep 16, 2026Manon VerbockhavenAutomatic DifferentiationReLU Neural Networks

    OCKHAM

  10. A Systematic Analysis of Automatic Differentiation versus Discretization-based Constraints for Physics-Informed PDE Solvers

    Sep 7, 2026Xing Guo, Hongwei Tang, Zewei Meng +3Automatic DifferentiationNeural PDE Solvers

    School of Systems Science and Engineering, Sun Yat-sen University, Guangzhou, China

  11. A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

    Sep 2, 2026Xilai Liang, Zhao ZhangPhysics-Informed MLAutomatic Differentiation

    Guangdong Technion–Israel Institute of Technology, Shantou, Guangdong, China · Research Centre for Mathematics and Interdisciplinary Sciences, Shandong University, Qingdao, Shandong Province, 266237, China

  12. Branch Geometry and Finite-Radius Sensitivity of Hard-ReLU Training

    Aug 31, 2026Xiaoyang Li, Runni Zhou, Xinghao YanDynamical SystemsAutomatic Differentiation

    College of Medicine and Biological Information Engineering Northeastern University, Shenyang, China

  13. Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond

    Aug 11, 2026Maciej J. Mikulski, Tadeusz UhlAutomatic DifferentiationNeural PDE Solvers

    AGH University of Krakow

  14. LazyHMC: Hamiltonian Monte Carlo Simulation for Lazy, Infinite Dimensional Probabilistic Programs

    Aug 9, 2026Maria-Nicoleta Crăciun, C. -H. Luke Ong, Tom Schrijvers +1Markov Chain Monte CarloAutomatic Differentiation

    University of Oxford, UK · Nanyang Technological University, Singapore, Singapore · KU Leuven, Leuven, Belgium

  15. Differentiate the Solver, Not the Equation: Reverse-Sweep Adjoints for Block Implicit Simulation

    Aug 9, 2026Lei Shu, Ying Jiang, Kui Wu +3Automatic DifferentiationDifferentiable Programming

    Stanford University · University of California, Los Angeles · LightSpeed Studios, Tencent America +1

  16. A Fortran General-Purpose Transpiler: Proof of Concept

    Jul 31, 2026Shivamshan Sivanesan, Kazem ArdanehCode TranslationAutomatic Differentiation

    1Climate Modeling Center, Sorbonne University, CNRS, IPSL, Paris, France

  17. Amortising Trajectory Optimisation for Residual MPC via Implicit Contact Differentiation

    Jul 27, 2026Daniel Layeghi, Thomas Corbères, Calum Arnott +4Trajectory OptimizationAutomatic Differentiation

    School of Informatics, University of Edinburgh, Scotland.

  18. FlashPDE: A Drop-In Fused Triton Operator Library for Neural PDE Solvers

    Jul 20, 2026Peiyu Zang, Bosen Xie, Ruoxiang Xu +1Automatic DifferentiationNeural PDE Solvers

    School of Mathematical Sciences, Beijing Normal University · 2Beijing University of Posts and Telecommunications

  19. Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning

    Jul 3, 2026Lucas ShenemanAutomatic DifferentiationDifferentiable Programming

    Institute for Interdisciplinary Data Sciences University of Idaho

  20. Mosaic: A Benchmark Suite for Differentiable Physics Solvers

    Jun 26, 2026Andrin Rehmann, Heiko Zimmermann, Dion HäfnerAutomatic DifferentiationPDE Solving

    Pasteur Labs, Brooklyn, NY, USA

  21. FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

    Jun 22, 2026Dikshant Kukreja, Kritarth Prasad, Avinash Anand +6Automatic DifferentiationBackpropagation

    Puch AI · Singapore Institute of Technology · Nanyang Technological University +2

  22. Automatic Differentiation from Scratch: How PyTorch Computes Gradients in Physics-Informed Neural Networks

    Jun 13, 2026Abdeladhim TahimiAutomatic DifferentiationBackpropagation

    CECA, Universidade Federal de Alagoas (UFAL), Rio Largo, Alagoas, 57100-000, Brazil

  23. Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter

    Jun 7, 2026Lucas ShenemanAutomatic DifferentiationProgram Synthesis

    Institute for Interdisciplinary Data Sciences University of Idaho

  24. Automatic-Differentiation-Enabled Finite Element Model Updating: Learning Constitutive Neural Networks from Structural Deformation Data

    May 22, 2026Matthias Knipper, Chenyi Ji, Malte Brand +1Automatic DifferentiationHyperelasticity

    Computational Mechanics in Medicine, Applied Medical Engineering, RWTH Aachen University Pauwelsstraße 20, 52074 Aachen, Germany · Institute for Continuum and Material Mechanics, Hamburg University of Technology, Eißendorfer Straße 42, 21073 Hamburg, Germany

  25. Archimedean Copula Inference via Taylor-Mode AD

    May 22, 2026Cambridge Yang, Dongdong LiParameter EstimationAutomatic Differentiation

    Independent · Harvard Medical School

  26. Differentiable Parameter Optimization for DAEs with State-Dependent Events

    May 6, 2026Ion Matei, Maksym Zhenirovskyy, Anthony WongDynamical SystemsAutomatic Differentiation

    Fujitsu Research of America

  27. Exact Higher-Order Derivatives for SE(3) via Analytical/AD Methods

    May 4, 2026Frank O. KuehnelAutomatic DifferentiationLie Group Methods

    Sigma Point Labs, Denver, CO, USA

  28. Floating-Point Networks with Automatic Differentiation Can Represent Almost All Floating-Point Functions and Their Gradients

    May 3, 2026Sejun Park, Yeachan Park, Geonho HwangAutomatic DifferentiationNeural Network Approximation Theory

    Department of Artificial Intelligence, Korea University, Seoul, 02841, Republic of Korea · Department of Mathematics and Statistics, Sejong University, Seoul, 05006, Republic of Korea · Department of Mathematical Sciences, Gwangju Institute of Science and Technology, Gwangju, 61005, Republic of Korea

  29. Block-Wise Differentiable Sinkhorn Attention: Tail-Refinement Gradients with a Gap-Aware Dustbin Bridge

    Apr 28, 2026Dylan FordeAutomatic DifferentiationTransformer Attention

    Independent Researcher

  30. VertAX: a differentiable vertex model for learning epithelial tissue mechanics

    Apr 8, 2026Alessandro Pasqui, Jim Martin Catacora Ocana, Anshuman Sinha +7Differentiable PhysicsDifferentiable Optimization

    Center for Interdisciplinary Research in Biology (CIRB), Collège de France, Université PSL, CNRS, INSERM, 75005 Paris, France. · Current address: Center for Integrative Biology (CBI), University of Toulouse, CNRS, Toulouse, France. · Center for Life Nano & Neuro Science, IIT, 00161, Rome, Italy. +3