Learning to Detect Symbolic Failure: Machine Learning and the Limits of Black-Scholes
Organizations: Stanford University
Abstract
We treat options pricing as a representation problem: can machine learning detect systematic deviations from Black-Scholes using 2.6M real option contracts? We compare three regimes: learned abstract embeddings (Kernel PCA), preserved domain structure (tree-based ensembles), and neural network validation. Tree-based methods outperform kernel dimensionality reduction by 21.5 percentage points (93.8% vs 72.3%), and domain-expert features (Greeks, moneyness) outperform engineered features. NN-based and BS-based deviation labels agree 99.9974% of the time, suggesting deviations reflect market structure rather than model artifact. We conclude that in domains with expert-designed symbolic features, preserving structure beats learning abstractions. We make no claim of exploitable mispricings.
Figures & tables
| Asset | Volatility | Contracts | Accuracy |
|---|---|---|---|
| AAPL | 25% IV | 407,224 | 91.3% |
| SPY | 15% IV | 1,341,246 | 98.2% |
| TSLA | 45% IV | 883,543 | 82.7% |
| Average | Mixed | 2,044,241 | 90.8% |
Quantitative Finance
Quantitative finance uses mathematical models, statistics, and machine learning to value assets, measure risk, and guide trading and portfolio decisions.
See all 191 papers