Geometric Identification in Predict-Then-Optimize Learning

Jiaxiao Xu; Changhong Mou; Keji Liu; Dinghua Xu; Yeyu Zhang

cs.LGSep 27, 2026

Geometric Identification in Predict-Then-Optimize Learning

Authors: Jiaxiao Xu, Changhong Mou, Keji Liu, Dinghua Xu, Yeyu Zhang

Organizations: Shanghai University of Finance and Economics · Utah State University

Abstract

Decision-focused surrogates can recover downstream decisions without identifying the quotient report. We characterize the equality set of the convex Smart Predict-then-Optimize surrogate (SPO+) population risk. Under central symmetry, the centered mean class is the unique Bayes minimizer exactly when every nonzero effective displacement makes the old optimizer leave the shifted optimal face with positive probability. This condition separates face crossing from selected-oracle disagreement and gives quantitative local coercivity. Without symmetry, strict crossing alone need not identify the mean; selection balance with reflected crossing restores quotient-report identification, and conditional versions extend the result to measurable predictors. These are population statements, without finite-sample report-recovery or generic transfer-regret guarantees. Closed-form mechanisms reproduce the analytic identities and rates. Portfolio, complete-matrix KuaiRec, and Energy/Storage studies measure predictive fidelity, shifted regret, and fitted-report geometry. A known data-generating process (DGP) companion retains their application geometries while isolating conditional-mean recovery and crossing, without testing the original observational assumptions.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Objective and model	Training and randomness
OLS / Ridge	Linear squared-report regression with ridge coefficient $10^{-3}$ ; the intercept is unpenalized when used.	Deterministic.
Linear SPO+	$\widehat{J}_{\mathrm{SPO+}}(B)$ , optimized by the SPO+ subgradient through the fixed oracle.	$T=40/100/40$ ; batch $=128$ /full/ $32$ ; $\eta_{0}=1/1/.01$ . Stochastic except full-batch KuaiRec.
SPO+ + report	$\widehat{J}_{\mathrm{hyb}}(B)$ ; thus $\lambda$ multiplies the squared report residual, not decision regret.	Same backbone and budget as Linear SPO+; $\lambda=.1$ .
PG weak report	Linear report trained with the perturbed-oracle decision gradient $\widehat{\mathbb{E}}[(w^{}(Bx+\sigma Z)-w^{}(c))/\sigma]x^{\top}$ . “Weak report” means that no squared report-fidelity term is included.	Stochastic: $n_{\rm pert}=3/16/3$ Gaussian perturbations/example; $\sigma=.01/.01/10$ , $\eta_{0}=.001/.01/10^{-5}$ , and the same $T$ ; batches $128/512/32$ . Portfolio uses a low-learning-rate setting chosen for numerical stability after validation screening; KuaiRec uses validation regret as the primary selection criterion.
Neural SPO+	KuaiRec-only MLP report $x\mapsto\hat{c}(x)$ trained with the SPO+ chain-rule subgradient.	Stochastic Adam; hidden widths $(128,64)$ , 100 epochs, batch $128$ , learning rate $10^{-3}$ , weight decay $10^{-4}$ .

Appendix

Table 1: Self-contained definitions of the empirical methods. The perturbed-oracle decision-gradient method (PG) uses Gaussian perturbations

Z

\sigma

is their report-space smoothing scale and

\eta

is the decayed step size

\eta_{0}/\sqrt{t+1}

at update step

t

Dataset	PG configuration $(\sigma,\eta_{0},n_{\rm pert})$	Validation RMSE	Test RMSE	Test shifted regret
Portfolio	$(.01,.001,3)$	$0.945\pm 0.029$	$0.878\pm 0.027$	$0.01517\pm 0.00009$
KuaiRec	$(.01,.01,16)$	$145.996\pm 1.274$	$150.60\pm 1.50$	$93.23\pm 1.11$

Appendix

Table 2: Final PG weak-report configurations. Values are five-seed means

\pm

sample SD; the reported settings were selected using validation screening and refit before the reported test evaluation.

Method	Report RMSE	$R_{0.10}$	$R_{0.12}$
OLS †	$0.1324\pm 0.0000$	$0.013924\pm 0.000000$	$0.015078\pm 0.000000$
SPO+	$0.2036\pm 0.0146$	$0.013977\pm 0.000050$	$0.015160\pm 0.000058$
SPO+ + report	$0.2114\pm 0.0175$	$0.013955\pm 0.000088$	$0.015120\pm 0.000109$
PG weak report	$0.878\pm 0.027$	$0.013972\pm 0.000071$	$0.015170\pm 0.000086$

Appendix

Table 3: Complete Portfolio five-seed means

\pm

sample SD.

R_{0.10}

is training-task regret and

R_{0.12}

is shifted regret.

\dagger

denotes a deterministic fit.

Method	Report RMSE	$R_{k=5}$	$R_{k=10}$	$R_{k=15}$
Ridge †	$58.92\pm 0.00$	$53.96\pm 0.00$	$71.00\pm 0.00$	$82.68\pm 0.00$
Linear SPO+ †	$71.73\pm 0.00$	$\mathbf{50.63}\pm 0.00$	$\mathbf{66.25}\pm 0.00$	$76.86\pm 0.00$
SPO+ + report †	$\mathbf{58.04}\pm 0.00$	$50.99\pm 0.00$	$66.71\pm 0.00$	$77.43\pm 0.00$
PG weak report	$150.60\pm 1.50$	$57.76\pm 0.56$	$78.77\pm 1.01$	$93.23\pm 1.11$
Neural SPO+ (pre-specified)	$100.53\pm 2.31$	$53.16\pm 0.20$	$70.72\pm 0.33$	$83.23\pm 0.25$
MLP (MSE, auxiliary)	$58.71\pm 0.02$	$52.03\pm 0.14$	$67.88\pm 0.28$	$78.67\pm 0.17$

Appendix

Table 4: Complete KuaiRec five-seed means

\pm

sample SD. The MLP report model is auxiliary; all other rows are pre-specified formal methods.

\dagger

marks deterministic linear fits.

Configuration	RMSE	$R_{5}$	$R_{10}$	$R_{15}$
50 epochs, 64	$73.35\pm 0.18$	$\mathbf{50.67}\pm 0.08$	$\mathbf{65.96}\pm 0.17$	$\mathbf{76.55}\pm 0.29$
50 epochs, $128\times 64$	$\mathbf{72.32}\pm 0.27$	$51.07\pm 0.04$	$66.58\pm 0.17$	$77.36\pm 0.23$
100 epochs, 64	$85.71\pm 0.73$	$51.99\pm 0.12$	$68.76\pm 0.29$	$80.36\pm 0.25$
100 epochs, $128\times 64$	$100.53\pm 2.14$	$53.16\pm 0.20$	$70.72\pm 0.33$	$83.23\pm 0.25$

Appendix

Table 5: Neural SPO+ capacity/epoch sensitivity check (five-seed means

\pm

sample SD), not an identification or strict-crossing test.

		LP relaxation			Binary scheduling
Method	RMSE	$s=0$	$s=0.5$	$s=1$	$s=0$	$s=0.5$	$s=1$
Ridge †	$31.65\pm 0.00$	$71.10\pm 0.00$	$52.96\pm 0.00$	$51.10\pm 0.00$	$71.11\pm 0.00$	$52.97\pm 0.00$	$51.09\pm 0.00$
SPO+	$111.30\pm 7.56$	$73.14\pm 29.12$	$58.89\pm 26.36$	$\mathbf{48.29}\pm 12.23$	$73.12\pm 29.12$	$58.88\pm 26.36$	$\mathbf{48.29}\pm 12.22$
SPO+ + report	$107.02\pm 6.23$	$72.59\pm 23.63$	$56.76\pm 22.72$	$50.16\pm 10.97$	$72.59\pm 23.63$	$56.76\pm 22.72$	$50.16\pm 10.97$
PG weak report	$74.67\pm 0.00$	$573.07\pm 102.57$	$700.72\pm 151.36$	$783.80\pm 197.00$	$573.07\pm 102.57$	$700.72\pm 151.36$	$783.79\pm 197.00$

Appendix

Table 6: Complete intercept-corrected Energy five-seed means

\pm

sample SD. Values are in thousands. Ridge is deterministic (

\dagger

Regime	Method	Excess	Regret	Q-error	Var
SC-on	Linear SPO+	$0.0146\pm 0.0064$	$0.3326\pm 0.0000$	$0.1217\pm 0.0330$	$0.0095$
	MLP-MSE	$0.0315\pm 0.0159$	$0.3326\pm 0.0000$	$0.1863\pm 0.0437$	$0.0261$
	MLP-SPO+	$0.0400\pm 0.0132$	$0.3326\pm 0.0000$	$0.2046\pm 0.0422$	$0.0290$
	MLP-SPO+ + MSE	$0.0388\pm 0.0119$	$0.3326\pm 0.0000$	$0.2039\pm 0.0446$	$0.0295$
Low mass	Linear SPO+	$0.0040\pm 0.0018$	$0.0360\pm 0.0000$	$0.0671\pm 0.0194$	$0.0026$
	MLP-MSE	$0.0205\pm 0.0051$	$0.0360\pm 0.0000$	$0.1481\pm 0.0262$	$0.0163$

Appendix

Table 7: Controlled neural identification experiment. “Excess”, “Regret”, and “Q-error” are five-seed means

\pm

sample SD; “Excess” is held-out SPO+ excess relative to the known conditional mean and “Q-error” is quotient-report error. “Var” is mean coordinatewise fitted report variance across seeds. The SC-off rows have equal observed decision regret but different fitted reports.

Case	$P(B_{tu})$	$\mathbb{E}[G_{tu}\mid B_{tu}]$	Rdiff
Box $S=[-1,1]^{2}$ (5 directions)	$1.00\ (1)$	$1.00\ (1)$	$2.01\ (2)$
Tri (4 directions)	$0.98$ to $1.01\ (1)$	$1.00\ (1)$	$1.93$ to $2.07\ (2)$
Lower density floor	$0.99\ (1)$	$0.99\ (1)$	$2.00\ (2)$
Flat density $e^{-1/\|c\|}$ (Exp. 2)	finite-grid slope $175.30$ at $t\approx 0.002$ ; diverges as $t\downarrow 0$

Appendix

Table 8: Fitted local log-log slopes (theory in parentheses). Box and Tri are the polytopes of Exp. 4; slopes are averaged over the crossing directions; the flat-density row reports a finite-grid slope from Exp. 2 and its asymptotic behavior.

Explore similar work

A Solver-Free Training Method for Predict-then-Optimize
Jun 17, 2026Beichen Wan, Mo LiuOptimization Modeling Surrogate Gradient
Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, NC, USA.
Smart predict-then-robustly-optimize
Jul 23, 2026Aakil Caunhye, Xuefei Lu, Belen Martin-BarraganRobust Optimization Predictive Uncertainty
The Curvature of Regret in Contextual Linear Optimization
Oct 1, 2026Konstantinos Ziliaskopoulos, Alexander Vinel, Alice E. SmithLinear Programming Regret
Auburn University Auburn, AL, USA · University of Alabama Tuscaloosa, AL, USA

Jun 17, 2026, stat.MLJ/K move · Enter open · S save

A Solver-Free Training Method for Predict-then-Optimize

Beichen Wan, Mo Liu

Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, NC, USA.

Optimization Modeling Surrogate Gradient Optimal Regret Scalability

Open paper AskPDF arXiv

We propose a scalable method for training prediction (machine learning) models in the predict-then-optimize paradigm, where model outputs serve as coefficients for a subsequent linear optimization task. Directly minimizing the empirical decision regret is intractable for linear programming and combinatorial optimization since the decision mapping is piecewise constant, and the gradients are zero almost everywhere. While existing methods address this by smoothing the differentiation process, they suffer from scalability issues, since a computationally expensive solver call is required for every gradient evaluation. To address this, we propose a decision-focused learning pipeline based on a measure transformation principle, which yields a new surrogate loss that is completely optimization-solver-free during training. We establish theoretical guarantees, including Fisher consistency and excess risk bounds. Empirically, our method achieves decision quality competitive with state-of-the-art methods while reducing training time by orders of magnitude.

Abstract

Explore similar work

A Solver-Free Training Method for Predict-then-Optimize

Smart predict-then-robustly-optimize

The Curvature of Regret in Contextual Linear Optimization