Automatic Differentiation

Automatic differentiation (AD) computes derivatives by applying the chain rule to the operations in a program, using reverse mode for many-input, few-output problems and forward or Taylor-mode methods for other derivative structures. Research develops higher-order and sparse differentiation, compiler and runtime systems, and implicit or solver-level adjoints that reduce memory and computation in settings such as neural networks, PDE solvers, and differentiable physical simulation. These methods support tasks including training physics-informed neural networks, parameter estimation, and optimization through event-driven or contact-rich systems. A central tension is that exact gradients of a computation can be expensive or may not reflect the desired physical sensitivity, so work compares AD with finite differences, spectral and discretization-based methods, and structure-aware derivative schemes.