Analog optical computers promise large efficiency gains for machine learning inference, yet no demonstration has moved beyond small-scale image benchmarks. We benchmark the analog optical computer (AOC) digital twin on mortgage approval classification from 5.84 million U.S. HMDA records and separate three sources of accuracy loss. On the original 19 features, the AOC reaches 94.6% balanced accuracy with 5,126 parameters (1,024 optical), compared with 97.9% for XGBoost; the 3.3 percentage-point gap narrows by only 0.5pp when the optical core is widened from 16 to 48 channels, suggesting an architectural rather than hardware limitation. Restricting all models to a shared 127-bit binary encoding drops every model to 89.4--89.6%, with an encoding cost of 8pp for digital models and 5pp for the AOC. Seven calibrated hardware non-idealities impose no measurable penalty. The three resulting layers of limitation (encoding, architecture, hardware fidelity) locate where accuracy is lost and what to improve next.
Figures & tables
Split
Samples
Unique groups
Class balance
Train
1,524,335
∼ 1,042,361
50.0 / 50.0%
Validation
191,838
∼ 130,295
50.3 / 49.7%
Test
190,607
∼ 130,295
49.8 / 50.2%
Table 1: Group-split dataset. All splits are near class-balanced.
Model
Features
Parameters
Optical
Test BACc (%)
XGBoost
19 raw
∼ 31,500
—
97.91±0.00
MLP-large
60 (one-hot)
24,578
—
97.46±0.04
MLP-small
60 (one-hot)
5,378
—
97.21±0.17
Ens-4x-AOC-48
60 (Ising)
21,510
9,216
95.14±0.46
Ens-4x-AOC-16
60 (Ising)
5,126
1,024
94.64±0.53
Ens-4x-AOC-16 (SimpleCell)
60 (Ising)
5,122
—
93.85±1.58
Table 2: Test balanced accuracy on 190,607 held-out mortgage applications, raw features (mean ± std over 3 seeds). Optical parameters are the subset executed on analog hardware.
Figure 1: Error-set overlap on raw features (XGBoost seed 42 vs AOC seed 0). XGBoost makes 3,971 errors; the AOC makes 9,792; only 3,577 are shared (Jaccard =0.351 ). On binarised features (Figure 2 ) the same pair shares over 90% of its errors (Jaccard =0.834 ). Binarisation collapses model diversity along with accuracy.
Model
Raw BACc (%)
Binarised BACc (%)
Δ (pp)
XGBoost
97.91
89.51
−8.40
MLP-large
97.46
89.57
−7.89
MLP-small
97.21
89.56
−7.65
Ens-4x-AOC-16
94.64
89.43
−5.21
Logistic regression
70.04
76.54
+6.50
Table 3: Effect of binarisation (mean over 3 seeds). All models evaluated on the same 190,607 test samples.
Figure 2: Error-set overlap on binarised features. Each model makes roughly 20,000 errors; 18,304 are shared (Jaccard =0.834 ). The models disagree on just 1.92% of the test set, having saturated the encoding ceiling.
Figure 3: Raw vs. binarised balanced accuracy for all model families. Binarisation costs 7.7 – 8.4pp for digital models and 5.2pp for the AOC. Logistic regression gains +6.5pp because one-hot binning linearises the feature space.
Figure 4: Decomposition of the accuracy gap between raw-feature XGBoost ( 97.9% ) and the binarised AOC ( 89.4% ). Architecture accounts for 3.3pp , encoding for 5.2pp , and hardware non-idealities for ∼0pp .
Figure 5: Balanced accuracy vs. parameter count. Filled circles: raw features; open squares: binarised. On raw features, AOC-16 and MLP-small have similar parameter counts ( ∼5,200 ) but a 2.6pp gap. In the binarised setting, all nonlinear models collapse to 89.4 – 89.6% regardless of capacity.