Automatic Differentiation

Ask about this topic

Momentum

6 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 26

All topics
CardsList

  1. Same Loss, Different Gradients

    Sep 30, 2026Ningkang Peng, Xiaoqian Peng, Yifan He +4Automatic DifferentiationDifferentiable Optimization

    Nanjing Normal University · Nanjing University of Chinese Medicine

  2. The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning

    Sep 27, 2026Ameya D. JagtapAutomatic DifferentiationPDE Surrogate Modeling

    Aerospace Engineering Department, Worcester Polytechnic Institute, Worcester, MA 01609, USA

  3. FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention

    Sep 21, 2026Anthony Givans, Michael Crawshaw, Mingrui LiuSoftmax AttentionAutomatic Differentiation

  4. Truncated automatic sparse differentiation for machine learning interatomic potentials

    Sep 17, 2026Marcel F. Langer, Adrian Hill, Michele CeriottiAutomatic DifferentiationMachine Learning Interatomic Potentials

    Laboratory of Computational Science and Modeling (COSMO), EPFL, Lausanne, Switzerland · Machine Learning Group, Technical University of Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany

  5. Rank and computation of the pathlifting Jacobian of a DAG ReLU network

    Sep 16, 2026Manon VerbockhavenAutomatic DifferentiationReLU Neural Networks

    OCKHAM

  6. A Systematic Analysis of Automatic Differentiation versus Discretization-based Constraints for Physics-Informed PDE Solvers

    Sep 7, 2026Xing Guo, Hongwei Tang, Zewei Meng +3Automatic DifferentiationNeural PDE Solvers

    School of Systems Science and Engineering, Sun Yat-sen University, Guangzhou, China

  7. A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

    Sep 2, 2026Xilai Liang, Zhao ZhangPhysics-Informed MLAutomatic Differentiation

    Guangdong Technion–Israel Institute of Technology, Shantou, Guangdong, China · Research Centre for Mathematics and Interdisciplinary Sciences, Shandong University, Qingdao, Shandong Province, 266237, China

  8. Hard-ReLU Gradient Descent Selects an Event-Free Sensitivity Limit

    Aug 31, 2026Xiaoyang Li, Runni ZhouDynamical SystemsAutomatic Differentiation

    College of Medicine and Biological Information Engineering Northeastern University, Shenyang, China

  9. Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond

    Aug 11, 2026Maciej J. Mikulski, Tadeusz UhlAutomatic DifferentiationNeural PDE Solvers

    AGH University of Krakow

  10. LazyHMC: Hamiltonian Monte Carlo Simulation for Lazy, Infinite Dimensional Probabilistic Programs

    Aug 9, 2026Maria-Nicoleta Crăciun, C. -H. Luke Ong, Tom Schrijvers +1Markov Chain Monte CarloAutomatic Differentiation

    University of Oxford, UK · Nanyang Technological University, Singapore, Singapore · KU Leuven, Leuven, Belgium

  11. Differentiate the Solver, Not the Equation: Reverse-Sweep Adjoints for Block Implicit Simulation

    Aug 9, 2026Lei Shu, Ying Jiang, Kui Wu +3Automatic DifferentiationDifferentiable Programming

    Stanford University · University of California, Los Angeles · LightSpeed Studios, Tencent America +1

  12. A Fortran General-Purpose Transpiler: Proof of Concept

    Jul 31, 2026Shivamshan Sivanesan, Kazem ArdanehCode TranslationAutomatic Differentiation

    1Climate Modeling Center, Sorbonne University, CNRS, IPSL, Paris, France

  13. Amortising Trajectory Optimisation for Residual MPC via Implicit Contact Differentiation

    Jul 27, 2026Daniel Layeghi, Thomas Corbères, Calum Arnott +4Trajectory OptimizationAutomatic Differentiation

    School of Informatics, University of Edinburgh, Scotland.

  14. FlashPDE: A Drop-In Fused Triton Operator Library for Neural PDE Solvers

    Jul 20, 2026Peiyu Zang, Bosen Xie, Ruoxiang Xu +1Automatic DifferentiationNeural PDE Solvers

    School of Mathematical Sciences, Beijing Normal University · 2Beijing University of Posts and Telecommunications

  15. Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning

    Jul 3, 2026Lucas ShenemanAutomatic DifferentiationDifferentiable Programming

    Institute for Interdisciplinary Data Sciences University of Idaho

  16. Mosaic: A Benchmark Suite for Differentiable Physics Solvers

    Jun 26, 2026Andrin Rehmann, Heiko Zimmermann, Dion HäfnerAutomatic DifferentiationPDE Solving

    Pasteur Labs, Brooklyn, NY, USA

  17. FORGE: Fused On-Register Gradient Elimination for Memory-Efficient LLM Training

    Jun 22, 2026Dikshant Kukreja, Kritarth Prasad, Avinash Anand +6Automatic DifferentiationBackpropagation

    Puch AI · Singapore Institute of Technology · Nanyang Technological University +2

  18. Automatic Differentiation from Scratch: How PyTorch Computes Gradients in Physics-Informed Neural Networks

    Jun 13, 2026Abdeladhim TahimiAutomatic DifferentiationBackpropagation

    CECA, Universidade Federal de Alagoas (UFAL), Rio Largo, Alagoas, 57100-000, Brazil

  19. Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter

    Jun 7, 2026Lucas ShenemanAutomatic DifferentiationProgram Synthesis

    Institute for Interdisciplinary Data Sciences University of Idaho

  20. Finite Element-Based Material Learning via Automatic Differentiation: Learning constitutive neural network models from full-field deformation data

    May 22, 2026Matthias Knipper, Chenyi Ji, Malte Brand +1Automatic DifferentiationHyperelasticity

    Computational Mechanics in Medicine, Applied Medical Engineering, RWTH Aachen University Pauwelsstraße 20, 52074 Aachen, Germany · Institute for Continuum and Material Mechanics, Hamburg University of Technology, Eißendorfer Straße 42, 21073 Hamburg, Germany

  21. Archimedean Copula Inference via Taylor-Mode AD

    May 22, 2026Cambridge Yang, Dongdong LiParameter EstimationAutomatic Differentiation

    Independent · Harvard Medical School

  22. Differentiable Parameter Optimization for DAEs with State-Dependent Events

    May 6, 2026Ion Matei, Maksym Zhenirovskyy, Anthony WongDynamical SystemsAutomatic Differentiation

    Fujitsu Research of America

  23. Exact Higher-Order Derivatives for SE(3) via Analytical/AD Methods

    May 4, 2026Frank O. KuehnelAutomatic DifferentiationLie Group Methods

    Sigma Point Labs, Denver, CO, USA

  24. Floating-Point Networks with Automatic Differentiation Can Represent Almost All Floating-Point Functions and Their Gradients

    May 3, 2026Sejun Park, Yeachan Park, Geonho HwangAutomatic DifferentiationNeural Network Approximation Theory

    Department of Artificial Intelligence, Korea University, Seoul, 02841, Republic of Korea · Department of Mathematics and Statistics, Sejong University, Seoul, 05006, Republic of Korea · Department of Mathematical Sciences, Gwangju Institute of Science and Technology, Gwangju, 61005, Republic of Korea

  25. Block-Wise Differentiable Sinkhorn Attention: Tail-Refinement Gradients with a Gap-Aware Dustbin Bridge

    Apr 28, 2026Dylan FordeAutomatic DifferentiationTransformer Attention

    Independent Researcher

  26. ADSEQ: A delay-aware autograd-compatible framework for spike-event delivery in SNNs

    Dec 5, 2025Lennart P. L. Landsmeer, Amirreza Movahedin, Said Hamdioui +1Automatic DifferentiationSpiking Neural Networks

    Department of Computer Engineering, Delft University of Technology, Netherlands · NeuroComputingLab, Department of Neuroscience, Erasmus Medical Center, Netherlands