Overview

Automatic differentiation (AD) computes derivatives by applying the chain rule to the operations in a program, using reverse mode for many-input, few-output problems and forward or Taylor-mode methods for other derivative structures. Research develops higher-order and sparse differentiation, compiler and runtime systems, and implicit or solver-level adjoints that reduce memory and computation in settings such as neural networks, PDE solvers, and differentiable physical simulation. These methods support tasks including training physics-informed neural networks, parameter estimation, and optimization through event-driven or contact-rich systems. A central tension is that exact gradients of a computation can be expensive or may not reflect the desired physical sensitivity, so work compares AD with finite differences, spectral and discretization-based methods, and structure-aware derivative schemes.

Latest papers

CardsList

36 papers3 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

  1. Block-Sparse Gauss-Newton Optimization of Finite Basis Physics-Informed Neural Networks: Sparse Automatic Differentiation Approach

    Nov 9, 2026Denis Korolev, Adrian Hill, Jeeeun LeeGauss-Newton OptimizationNeural PDE Solvers

  2. Same Physics, Different Gradients: A State-of-the-Art Report on Differentiable Physics Simulation

    Nov 9, 2026Lei Shu, Zizhou Huang, Yunuo Chen +15adjoint methodDifferentiable Physics

  3. Automatic Differentiation with Constraints in Forward and Inverse Problems

    Oct 20, 2026Nikolay Yavich, Alexander Ryabov, Viacheslav Naumov +4PDE SolvingPDE Inverse Problems

  4. Coordinate-Free Automatic Differentiation

    Oct 8, 2026Alex Guerra, Joshua Aduol, Yaqian Tang +3Automatic DifferentiationDifferentiable Programming

  5. asdex: Automatic Sparse Differentiation in JAX

    Oct 8, 2026Adrian Hill, Guillaume DalleAutomatic DifferentiationStructured Sparsity

  6. Recursive Improvement of a Differentiable Scientific Software Ecosystem

    Oct 3, 2026Pengcheng Hou, Xiaojun Tan, Sihan Hu +5Differentiable ProgrammingAutomatic Differentiation

  7. Same Loss, Different Gradients

    Sep 30, 2026Ningkang Peng, Xiaoqian Peng, Yifan He +4Automatic DifferentiationDifferentiable Optimization

  8. The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning

    Sep 27, 2026Ameya D. JagtapAutomatic DifferentiationPDE Surrogate Modeling

  9. Truncated automatic sparse differentiation for machine learning interatomic potentials

    Sep 17, 2026Marcel F. Langer, Adrian Hill, Michele CeriottiAutomatic DifferentiationMachine Learning Interatomic Potentials

  10. Rank and computation of the pathlifting Jacobian of a DAG ReLU network

    Sep 16, 2026Manon VerbockhavenAutomatic DifferentiationReLU Neural Networks

  11. A Systematic Analysis of Automatic Differentiation versus Discretization-based Constraints for Physics-Informed PDE Solvers

    Sep 7, 2026Xing Guo, Hongwei Tang, Zewei Meng +3Automatic DifferentiationNeural PDE Solvers

  12. A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

    Sep 2, 2026Xilai Liang, Zhao ZhangPhysics-Informed MLAutomatic Differentiation

  13. Branch Geometry and Finite-Radius Sensitivity of Hard-ReLU Training

    Aug 31, 2026Xiaoyang Li, Runni Zhou, Xinghao YanDynamical SystemsAutomatic Differentiation

  14. LazyHMC: Hamiltonian Monte Carlo Simulation for Lazy, Infinite Dimensional Probabilistic Programs

    Aug 9, 2026Maria-Nicoleta Crăciun, C. -H. Luke Ong, Tom Schrijvers +1Markov Chain Monte CarloAutomatic Differentiation

  15. Differentiate the Solver, Not the Equation: Reverse-Sweep Adjoints for Block Implicit Simulation

    Aug 9, 2026Lei Shu, Ying Jiang, Kui Wu +3Automatic DifferentiationDifferentiable Programming

  16. A Fortran General-Purpose Transpiler: Proof of Concept

    Jul 31, 2026Shivamshan Sivanesan, Kazem ArdanehCode TranslationAutomatic Differentiation

  17. Amortising Trajectory Optimisation for Residual MPC via Implicit Contact Differentiation

    Jul 27, 2026Daniel Layeghi, Thomas Corbères, Calum Arnott +4Trajectory OptimizationAutomatic Differentiation

  18. FlashPDE: A Drop-In Fused Triton Operator Library for Neural PDE Solvers

    Jul 20, 2026Peiyu Zang, Bosen Xie, Ruoxiang Xu +1Automatic DifferentiationNeural PDE Solvers

  19. Mosaic: A Benchmark Suite for Differentiable Physics Solvers

    Jun 26, 2026Andrin Rehmann, Heiko Zimmermann, Dion HäfnerAutomatic DifferentiationPDE Solving

  20. FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

    Jun 22, 2026Dikshant Kukreja, Kritarth Prasad, Avinash Anand +6Automatic DifferentiationBackpropagation

  21. Automatic Differentiation from Scratch: How PyTorch Computes Gradients in Physics-Informed Neural Networks

    Jun 13, 2026Abdeladhim TahimiAutomatic DifferentiationBackpropagation

  22. Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter

    Jun 7, 2026Lucas ShenemanAutomatic DifferentiationProgram Synthesis

  23. Automatic-Differentiation-Enabled Finite Element Model Updating: Learning Constitutive Neural Networks from Structural Deformation Data

    May 22, 2026Matthias Knipper, Chenyi Ji, Malte Brand +1Automatic DifferentiationHyperelasticity

  24. Archimedean Copula Inference via Taylor-Mode AD

    May 22, 2026Cambridge Yang, Dongdong LiParameter EstimationAutomatic Differentiation

  25. Differentiable Parameter Optimization for DAEs with State-Dependent Events

    May 6, 2026Ion Matei, Maksym Zhenirovskyy, Anthony WongHybrid Dynamical SystemsDynamical Systems

  26. Floating-Point Networks with Automatic Differentiation Can Represent Almost All Floating-Point Functions and Their Gradients

    May 3, 2026Sejun Park, Yeachan Park, Geonho HwangAutomatic DifferentiationNeural Network Approximation Theory