Learning to Detect Symbolic Failure: Machine Learning and the Limits of Black-Scholes

Juli Huang; Jake Cheng; Rupert Lu

cs.LGAug 24, 2026

Learning to Detect Symbolic Failure: Machine Learning and the Limits of Black-Scholes

Authors: Juli Huang, Jake Cheng, Rupert Lu

Organizations: Stanford University

Abstract

We treat options pricing as a representation problem: can machine learning detect systematic deviations from Black-Scholes using 2.6M real option contracts? We compare three regimes: learned abstract embeddings (Kernel PCA), preserved domain structure (tree-based ensembles), and neural network validation. Tree-based methods outperform kernel dimensionality reduction by 21.5 percentage points (93.8% vs 72.3%), and domain-expert features (Greeks, moneyness) outperform engineered features. NN-based and BS-based deviation labels agree 99.9974% of the time, suggesting deviations reflect market structure rather than model artifact. We conclude that in domains with expert-designed symbolic features, preserving structure beats learning abstractions. We make no claim of exploitable mispricings.

Figures & tables

Figure 1: Volatility smile: implied volatility varies with moneyness, directly contradicting Black-Scholes’ constant volatility assumption. This systematic deviation is what our models learn to detect.

Asset	Volatility	Contracts	Accuracy
AAPL	25% IV	407,224	91.3%
SPY	15% IV	1,341,246	98.2%
TSLA	45% IV	883,543	82.7%
Average	Mixed	2,044,241	90.8%

Table 1: Gradient Boosting accuracy by asset (2,044,241 labeled contracts).

Figure 2: Model comparison across three representational regimes. Tree-based methods (which preserve domain structure) outperform kernel PCA (which learns abstract embeddings) by 21.5 percentage points. Neural network validation (99.9974% agreement) confirms that deviations are robust to the choice of pricing model.

Figure 3: Feature importance breakdown: domain-expert features (Greeks, moneyness, implied volatility) dominate. Engineered features (interaction terms) contribute negligibly, suggesting domain expertise has already saturated the signal.

Quantitative Finance

Quantitative finance uses mathematical models, statistics, and machine learning to value assets, measure risk, and guide trading and portfolio decisions.

See all 191 papers

Inverse Learning of Latent Risk-Neutral Densities from Irregular Option Quotes
Jul 29, 2026Lennon J. Shikhman, Michael Galarnyk, Aadi Dash +1Quantitative Finance Deep Operator Networks
College of Computing, Georgia Institute of Technology, USA · College of Engineering, Georgia Institute of Technology, USA · Department of Mathematics and Systems Engineering, Florida Institute of Technology, USA
PIVOT: Bridging Black-Scholes Implied-Volatility and Price Objectives via Differentiable Jäckel Operator
Jun 4, 2026Raeid Saqur, Yannick Limmer, Anastasis Kratsios +2Quantitative Finance Differentiable Programming
Mathematical Institute, University of Oxford · Vector Institute for AI · DRW +1
Neural Networks with Local Converging Inputs for Efficient Options Pricing Models
Aug 3, 2026Harris Cobb, Wenbo Hao, Yingjie LiuNeural Surrogate Modeling Neural PDE Solvers
School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332

Jul 29, 2026J/K move · Enter open · S save

Inverse Learning of Latent Risk-Neutral Densities from Irregular Option Quotes

Lennon J. Shikhman, Michael Galarnyk, Aadi Dash, Nicholas A. Welsh

College of Computing, Georgia Institute of Technology, USA · College of Engineering, Georgia Institute of Technology, USA · Department of Mathematics and Systems Engineering, Florida Institute of Technology, USA

Quantitative Finance Deep Operator Networks Density Estimation Neural Operators Inverse Problems Finance

Open paper AskPDF arXiv

Accurate option prices do not imply accurate recovery of the latent risk-neutral density. We study this distinction with two complementary benchmarks. A controlled benchmark exposes simulator-truth densities for latent evaluation, while a chronological NIFTY benchmark tests only held-out market prices. A two-component lognormal mixture has the lowest aggregate price,

L^1

, Wasserstein, and fixed-tail errors on the synthetic benchmark. Learned operators retain narrower strengths: DeepONet reduces 1% quantile and variance error by 39.0% and 34.6% relative to the mixture, and a quote transformer reduces

L^1

by 16.4% on the structurally misspecified Merton family. A numerical conditioning analysis explains why these rankings can differ: after enforcing mass and forward constraints, 95 of 126 pricing directions are numerically null, and two densities separated by

L^1 = 0.061

produce identical prices on the covered strikes. On 524 held-out NIFTY calls, validation-selected test-time adaptation reduces DeepONet RMSE by 28.3%, but per-expiry mixture and SVI fits remain much more accurate. The evidence supports target-dependent inductive bias, not a universal winner.

Abstract

Quantitative Finance

Explore similar work

Inverse Learning of Latent Risk-Neutral Densities from Irregular Option Quotes

PIVOT: Bridging Black-Scholes Implied-Volatility and Price Objectives via Differentiable Jäckel Operator

Neural Networks with Local Converging Inputs for Efficient Options Pricing Models