OptiRepair: Closed-Loop Diagnosis and Repair of Supply Chain Optimization Models with LLM Agents

Ruicheng Ao; David Simchi-Levi; Xinshang Wang

cs.AIFeb 23, 2026

OptiRepair: Closed-Loop Diagnosis and Repair of Supply Chain Optimization Models with LLM Agents

Authors: Ruicheng Ao, David Simchi-Levi, Xinshang Wang

Organizations: Massachusetts Institute of Technology. · Alibaba Group.

Abstract

Supply chain optimization models frequently become infeasible because of modeling errors. Diagnosis and repair require scarce OR expertise: analysts must interpret solver diagnostics, trace root causes across echelons, and fix formulations without sacrificing operational soundness. Whether AI agents can perform this task remains untested. We decompose this task into two phases: a domain-agnostic feasibility phase that iteratively repairs any LP using IIS-guided diagnosis, and a domain-specific validation phase that enforces five rationality checks grounded in inventory theory. We test 22 API models from seven families on 976 multi-echelon supply chain problems and train two 8B-parameter models with self-taught reasoning and solver-verified rewards. The trained models reach 81.7% Rational Recovery Rate (RRR) -- the fraction of problems resolved to both feasibility and operational rationality -- versus 42.2% for the best API model and 21.3% on average. The gap concentrates in Phase 1 repair, where API models average 27.6% recovery rate versus 97.2% for trained models. Two gaps separate current AI from reliable model repair: solver interaction, as API models restore only 27.6% of infeasible formulations; and operational rationale, as roughly one in four feasible repairs violate supply chain theory. Each gap requires a different intervention -- targeted training closes the solver interaction gap, while explicit specification as solver-verifiable checks closes the rationality gap. For organizations adopting AI in operational planning, formalizing what 'rational' means in their context is the higher-return investment.

Figures & tables

Configuration	RRR
Trained 8B pipeline (best)	81.7%
Best API (GPT-5.2 / Gemini 2.5 Pro)	42.2%
Average API (22 models, 7 families)	21.3%

Table 1: Central result on OptiRepair-SC (284-problem test set). RRR : fraction of problems where the model restores feasibility and produces an operationally rational solution.

Framework	Task	Oracle	Multi-step	Self-Corr.	Rationality
NL4Opt ( Ramamonjison et al., 2023 )	NL $\to$ LP	—	—	—	—
OptiBench ( Yang et al., 2024 )	Formulation	—	—	—	—
MAMO ( Huang et al., 2024 )	Complex LP	—	—	—	—
ORLM ( Huang et al., 2025 )	Formulation	—	—	—	—
LEAN-LLM-OPT ( Liang et al., 2026 )	Formulation	Solver	✓	—	—
AIM-Bench ( Zhao et al., 2025 )	Inventory	Closed-form	—	—	✓

Table 2: Positioning against optimization-agent benchmarks. OptiRepair-SC is the first to evaluate both iterative diagnosis-and-repair and operational rationality in a closed-loop supply chain setting.

Type	Error Name	Operational Meaning	Difficulty
ME-1	Demand Inflation	Overstated downstream demand	Hard
ME-2	Lead Time Error	Misspecified replenishment lead time	Hard
ME-3	Balance Violation	Impossible inventory balance state	Easy
ME-4	Capacity Reduction	Production capacity below demand	Hard
ME-5	Cost Structure Error	Non-monotonic holding costs	Easy
ME-6	Bullwhip Amplification	Forced upstream order amplification	Medium

Table 3: Supply chain error classification. Ten error types ( ME-1 – ME-10 ) represent common modeling mistakes in multi-echelon inventory planning.

Figure 1: Two-phase closed-loop architecture of OptiRepair . Phase I (domain-agnostic) iteratively diagnoses and repairs an infeasible LP via solver IIS feedback. Upon reaching Optimal , Phase II (domain-specific) validates operational correctness through a replaceable domain oracle—implemented here with five supply chain rationality checks.

Figure 2: Phase I repair interaction. The agent receives an infeasible supply chain LP (Gurobi), uses GET_IIS to diagnose conflicting constraints, and repairs the capacity bound. When the solver returns Optimal , the episode transitions to Phase II. Full system prompt and transcript appear in Appendix D .

Figure 3: Phase II rationality repair interaction. The rationality oracle detects that the Phase I repair created a cost-consistency violation: the high-cost retailer holds more inventory than the cheaper warehouse. The agent corrects the objective coefficient; the solver re-optimizes and the oracle confirms all five checks pass.

Figure 4: OptiRepair-SC construction pipeline. \raisebox{-.9pt}{\scriptsize1}⃝ A multi-echelon supply chain generator produces valid LPs with 2–5 echelons and 12–24 periods. \raisebox{-.9pt}{\scriptsize2}⃝ A saboteur injects one of ten error types, grouped into five categories (demand/timing, balance/capacity, cost/structure, coefficient/sign, constraint/index). \raisebox{-.9pt}{\scriptsize3}⃝ Gurobi confirms the sabotaged model is infeasible (or Optimal -but-irrational for ME-5 ). \raisebox{-.9pt}{\scriptsize4}⃝ The ground-truth fix is verified against five rationality checks. \raisebox{-.9pt}{\scriptsize5}⃝ Validated instances form OptiRepair-SC (976 problems, disjoint train/test splits).

Figure 5: OptiSTaR training pipeline. Top : Phase I iterates through beam search exploration (

K\!=\!32

candidates), STaR distillation, and GRPO refinement; three iterations raise

\text{RR@}{5}

from 44.4% to 75.7%. Gurobi provides IIS feedback for beam search and outcome rewards for GRPO . Middle right : Phase II trains independently on oracle-generated data (SFT) refined by solver-based GRPO . Bottom : The two independently trained 8B models combine into the final pipeline (

\text{RRR}=81.7\%

#	Model	RR	RRR	P2Pass	Steps	#	Model	RR	RRR	P2Pass	Steps
1	GPT-5.2	53.5	42.2	79.0	14.0	12	Claude Haiku 4.5	20.8	19.0	91.5	7.4
2	Gemini 2.5 Pro	53.9	42.2	78.4	9.3	13	GPT-o3	19.7	19.0	96.4	7.5
3	DeepSeek-R1-0528	53.2	37.0	69.5	13.8	14	GPT-4.1	18.0	17.6	98.0	5.9
4	Gemini 2.5 Flash	45.1	32.0	71.1	9.5	15	GPT-4.1 Mini	17.2	17.2	100.0	6.6
5	Qwen2.5-7B	41.2	31.0	75.2	8.1	16	GPT-5 Mini	18.7	16.9	90.6	6.8
6	Claude Opus 4.5	33.5	30.3	90.5	16.6	17	GPT-o1	14.9	13.5	90.5	4.4

Table 4: OptiRepair-SC results: 22 API models on the supply chain closed-loop pipeline (284 test problems). Models ranked by Rational Recovery Rate ( RRR ).

Model	Arch.	P1 RR	RRR	ME-2	ME-4	ME-5	Steps	Tokens
Qwen3-8B ( OptiSTaR )	Qwen	97.2	81.7	64.5	61.3	100.0	5.2	3,439
Llama-3.1-8B ( OptiSTaR )	Llama	97.2	80.3	67.7	74.2	100.0	4.3	3,180
Best API (GPT-5.2)	—	53.5	42.2	41.9	38.7	100.0	14.0	20,730
Avg. API (22 models)	—	27.6	21.3	3.7	4.7	87.3	7.8	30,078

Table 5: Trained 8B models vs. API baselines on OptiRepair-SC (284 test problems). Domain-specialized 8B models outperform all 22 frontier API models by 38–40 percentage points in RRR , while using 2–3

\times

fewer steps and 6–19

\times

fewer tokens.

Figure 6: Rational Recovery Rate ( RRR ) for 22 API models and two trained 8B pipelines ( Qwen3-8B and Llama-3.1-8B ) on OptiRepair-SC (284-problem test set). The vertical dashed line marks the API average (21.3%). Both trained models outperform the best API by

\geq

38 percentage points.

Figure 7: Phase 1 Recovery Rate vs. Phase 2 Pass Rate for all 22 API models and two trained 8B pipelines. Bubble size

\propto

RRR . API models cluster upper-left (high P2Pass , low RR ); trained models reach the upper-right corner. The vertical gap between clusters—roughly 70 percentage points in RR —confirms that Phase 1 repair drives the overall performance difference.

Qwen3-8B pipeline (8B)
P1 Model	P2 Model	P1 RR (%)	RRR (%)
Qwen3-8B OptiSTaR	Qwen3-8B P2 OptiSTaR	97.2	81.7
Qwen3-8B OptiSTaR	Claude Opus 4.5	97.2	69.4
Qwen3-8B OptiSTaR	GPT-5.2	96.8	68.7
Claude Opus 4.5	Qwen3-8B P2 OptiSTaR	55.6	45.1
GPT-5.2	Qwen3-8B P2 OptiSTaR	63.0	55.6

Table 6: Component ablation on OptiRepair-SC (284 test problems). Phase 1 repair is the critical bottleneck: replacing the trained P1 model with API alternatives reduces RRR by 22–40 percentage points. The pattern holds across both architectures.

Figure 8: Repair efficiency: average steps vs. average tokens per problem for all 24 models. Bubble size

\propto

RRR . Trained 8B models cluster in the bottom-left corner—fewer steps, fewer tokens, higher RRR —while the highest- RRR API models (GPT-5.2, DeepSeek-R1) consume the most resources.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Injection Mechanism	IIS Signature	Difficulty
ME-1	Add systematic offset ( $3$ – $6\times$ demand mean) to demand propagation at one echelon	Multiple demand_prop constraints + capacity bounds	Hard
ME-2	Remove arrival term from all inventory balance constraints at target echelon	Multiple inv_balance constraints + backorder caps	Hard
ME-3	Force net inventory ( $I_{n,t}-B_{n,t}$ ) to value below achievable minimum	Single inv_balance constraint with impossible target	Easy
ME-4	Set order capacity to $0.02$ – $0.1\times$ demand mean at retailer	Multiple capacity constraints + inventory balance	Hard
ME-5	Inflate upstream holding cost to $1.5$ – $3.0\times$ downstream cost	None ( Optimal but irrational); detected by rationality oracle	Easy
ME-6	Add constraints forcing $x_{n,t}\geq x_{n-1,t-1}+\text{offset}$ exceeding capacity	Multiple bullwhip_force constraints + supply cap	Medium

Appendix

Table 7: Error type injection specifications.

Error	Base-stock	Bullwhip	Inv. alloc.	Cost consist.	Order smooth.	Exclusion rationale
ME-1	✓	–	✓	✓	–	Demand inflation creates legitimate order variance
ME-2	✓	–	✓	✓	–	Lead-time repair creates legitimate order variance
ME-3	✓	–	✓	✓	–	Balance repair creates legitimate order variance
ME-4	✓	–	✓	✓	–	Capacity change creates legitimate order variance
ME-5	–	–	–	✓	–	Cost-only error; cost consistency detects the root cause
ME-6	–	–	–	–	✓	Bullwhip error; only order smoothing is artifact-free

Appendix

Table 8: Oracle check applicability by error type. A check mark (✓) indicates the check is applied; a dash (–) indicates exclusion. The “Exclusion rationale” column explains why excluded checks produce false failures for that error type.

Model	ME-1	ME-2	ME-3	ME-4	ME-5	ME-6	ME-7	ME-8	ME-9	ME-10
Qwen3-8B (trained)	85.2	64.5	96.7	61.3	100	89.3	91.7	88.0	96.4	86.7
Llama-3.1-8B (trained)	81.5	67.7	93.3	74.2	100	85.7	87.5	84.0	92.9	83.3
GPT-5.2	7.4	41.9	73.3	38.7	100	7.1	50.0	36.0	57.1	6.7
Gemini 2.5 Pro	0.0	6.5	63.3	6.5	100	60.7	50.0	60.0	57.1	23.3
DeepSeek-R1	3.7	12.9	76.7	9.7	100	0.0	58.3	60.0	42.9	10.0
Gemini 2.5 Flash	0.0	3.2	66.7	9.7	100	21.4	41.7	44.0	21.4	13.3

Appendix

Table 9: Per-error-type RRR (%) on OptiRepair-SC for trained models and top-6 API baselines. ME-5 (cost structure) is universally solvable; ME-2 (lead time) and ME-4 (capacity) separate trained models from API baselines.

Rank	Model	ME-1	ME-2	ME-3	ME-4	ME-5	ME-6	ME-7	ME-8	ME-9	ME-10	Overall
1	GPT-5.2	7.4	41.9	73.3	38.7	100	7.1	50.0	36.0	57.1	6.7	42.2
2	Gemini 2.5 Pro	0.0	6.5	63.3	6.5	100	60.7	50.0	60.0	57.1	23.3	42.2
3	DeepSeek-R1-0528	3.7	12.9	76.7	9.7	100	0.0	58.3	60.0	42.9	10.0	37.0
4	Gemini 2.5 Flash	0.0	3.2	66.7	9.7	100	21.4	41.7	44.0	21.4	13.3	32.0
5	Qwen2.5-7B	40.7	0.0	66.7	9.7	90.0	3.6	29.2	36.0	0.0	33.3	31.0
6	Claude Opus 4.5	3.7	6.5	70.0	3.2	100	35.7	37.5	20.0	25.0	0.0	30.3

Appendix

Table 10: RRR (%) by error type for all 22 API models and the trained Qwen3-8B pipeline on OptiRepair-SC test set. Bold = highest API score per error type.

Parameter	Range	Distribution
Number of echelons $N$	$\{2,3,4,5\}$	Uniform
Planning periods $T$	$\{12,16,20,24\}$	Uniform
Topology	Serial	Fixed
Holding cost $h_{n}$	$[1,10]$ per unit/period	Uniform
Backorder cost $b_{n}$	$[5,50]$ per unit/period	Uniform
$h$ -to- $b$ ratio	$1{:}2$ to $1{:}10$	Derived

Appendix

Table 11: Parameter ranges for OptiRepair-SC instance generation.

	Train	Test	Total
Total problems	692	284	976
ME-1 (Demand Inflation)	78	27	105
ME-2 (Lead Time Error)	89	31	120
ME-3 (Balance Violation)	87	30	117
ME-4 (Capacity Reduction)	89	31	120
ME-5 (Cost Structure Error)	71	30	101

Appendix

Table 12: Dataset split statistics for OptiRepair-SC .

Supply Chain Optimization

See all 19 papers

TACIT: Optimization Models that Learn from Their Mistakes
Sep 29, 2026Maxime Bouscary, Marco Molinaro, Sirui Li +3Large Language Model-Guided Optimization
Massachusetts Institute of Technology, Cambridge, MA, USA · Microsoft Research, Redmond, WA, USA
SAILOR: Solver-Assisted Interactive LLM-based Optimization Recovery
Sep 12, 2026Shaghayegh Sadeghi, Stephen L. Smith, David C. Del Rey Fern'andezLarge Language Model-Guided Optimization Natural Language to Optimization Modeling
Future Cities Institute, University of Waterloo, ON, Canada · Electrical and Computer Engineering, University of Waterloo, ON, Canada · Department of Applied Mathematics, University of Waterloo, ON, Canada
ReflectiChain: Epistemic Grounding in LLM-Driven World Models for Supply Chain Resilience
Jun 9, 2026Jia LuoConstrained RL LLM-Guided RL
School of Foreign Languages, Huazhong University of Science and Technology, Wuhan 430074, China.

Sep 29, 2026J/K move · Enter open · S save

TACIT: Optimization Models that Learn from Their Mistakes

Maxime Bouscary, Marco Molinaro, Sirui Li, Saurabh Amin+2

Massachusetts Institute of Technology, Cambridge, MA, USA · Microsoft Research, Redmond, WA, USA

Large Language Model-Guided Optimization

Open paper AskPDF arXiv

Real-world optimization problems are difficult to model accurately because many objectives and constraints reside in domain experts' tacit knowledge, making them hard to formalize. As a result, optimization models often contain miscalibrated objectives, missing constraints, or omitted decision variables, leading to solutions that fail to reflect operational realities. We address this challenge by automatically repairing misspecified formulations using historical data consisting of past solutions and subsequent user overrides. Traditional approaches such as inverse optimization and constraint learning tend to overfit sparse data and produce complex formulations. Our central idea is to combine the reasoning capabilities and prior knowledge of LLMs with the formal grounding provided by optimization. We realize this idea through two complementary paradigms. Top-down, an LLM proposes structural repairs, including new constraints and variables, whose numerical parameters are calibrated and validated through optimization. Bottom-up, optimization infers cuts from observed decisions, which the LLM contextualizes into interpretable, generalizable modeling constraints. We evaluate our approach on 38 misspecification scenarios spanning nine classes of optimization problems, several drawn from real-world applications, and show that TACIT can repair 78.9% of them (vs. 60.5% for the best baseline).

Abstract

Supply Chain Optimization

Explore similar work

TACIT: Optimization Models that Learn from Their Mistakes

SAILOR: Solver-Assisted Interactive LLM-based Optimization Recovery

ReflectiChain: Epistemic Grounding in LLM-Driven World Models for Supply Chain Resilience