SAILOR: Solver-Assisted Interactive LLM-based Optimization Recovery

Shaghayegh Sadeghi; Stephen L. Smith; David C. Del Rey Fern'andez

cs.AISep 12, 2026

SAILOR: Solver-Assisted Interactive LLM-based Optimization Recovery

Authors: Shaghayegh Sadeghi, Stephen L. Smith, David C. Del Rey Fern'andez

Organizations: Future Cities Institute, University of Waterloo, ON, Canada · Electrical and Computer Engineering, University of Waterloo, ON, Canada · Department of Applied Mathematics, University of Waterloo, ON, Canada

Abstract

Natural-language descriptions of optimization problems may be incomplete or vague about numerical information that a solver requires, including costs, capacities, demands, bounds, and penalties. A language model can translate the description into code, but when a required value is absent it must either stop or guess. We present SAILOR, a proof-of-concept system that detects such unsupported numerical choices, asks the user targeted follow-up questions, and updates the optimization model before returning a solution. Questions are prioritized using uncertainty and solver-derived estimates of how strongly each missing value affects the current model. We evaluate the pipeline on 1,723 instances from seven masked benchmarks using an idealized simulator that returns ground-truth values. Exact objective-value agreement ranges from 27.0% to 87.6% across datasets, with 1.4--5.7 questions per instance on average. These results establish feasibility under controlled branch-and-reveal feedback; they do not measure performance with human users or general structural model repair. Code is available at: https://github.com/sshaghayeghs/SAILOR.

Figures & tables

Figure 1: Parameter sensitivity under uncertainty. In panel (a), perturbing a low-sensitivity parameter across its current belief range barely moves the optimal objective. In panel (b), the same kind of sweep over a high-sensitivity parameter swings the objective sharply. That gap is why we use solver-based sensitivity to decide which parameters to ask about first.

Figure 2: The three conceptual belief-update operations. Each panel shows the prior (dashed gray) and posterior (solid). Panel (a) is a pure binary update, panel (b) an exact update, and panel (c) an optional soft refinement update. Our experiments exercise (b) and (c); see Section 6.1 .

Dataset	Instances	Avg Params	Source	Description
NL4OPT	228	3.5	Ramamonjison et al. (2023)	Simple two-variable linear programs.
NL4LP	178	3.9	Xiao et al. (2025)	Linear programs of moderate complexity.
ReSocratic	403	$\sim\!3$	Xiao et al. (2025)	Socratic-style optimization problems.
EasyLP	652	$\sim\!4$	( Huang et al., 2024 )	Small linear programs from MAMO.
ComplexLP	211	8.4	( Huang et al., 2024 )	Complex LPs, up to $87$ parameters.
IndustryOR	42	5.8	Xiao et al. (2025)	Industrial scheduling, $15$ – $30$ parameters.

Table 1: Datasets used in our evaluation. Avg Params is the mean number of numerical parameters per instance, taken from the ground-truth optimization model.

Metric	NL4LP	EasyLP	NL4OPT	ReSocratic	IndustryOR	ComplexOR	ComplexLP
Exact	87.6%	79.8%	78.1%	77.9%	64.3%	50.0%	27.0%
No-interact.	16.3 $\downarrow$ 71.3	34.2 $\downarrow$ 45.6	13.2 $\downarrow$ 64.9	13.1 $\downarrow$ 64.8	15.0 $\downarrow$ 49.3	5.6 $\downarrow$ 44.4	5.7 $\downarrow$ 21.3
$\leq$ 1%	88.2% $\uparrow$ 0.6	80.2% $\uparrow$ 0.4	80.7% $\uparrow$ 2.6	81.6% $\uparrow$ 3.7	64.3% $\uparrow$ 0.0	50.0% $\uparrow$ 0.0	32.2% $\uparrow$ 5.2
$\leq$ 5%	90.4% $\uparrow$ 2.8	82.2% $\uparrow$ 2.4	85.1% $\uparrow$ 7.0	83.9% $\uparrow$ 6.0	64.3% $\uparrow$ 0.0	55.6% $\uparrow$ 5.6	53.1% $\uparrow$ 26.1
$\leq$ 10%	91.6% $\uparrow$ 4.0	83.1% $\uparrow$ 3.3	87.3% $\uparrow$ 9.2	86.6% $\uparrow$ 8.7	66.7% $\uparrow$ 2.4	61.1% $\uparrow$ 11.1	62.6% $\uparrow$ 35.6
Gap	5.5%	12.0%	8.1%	8.0%	22.7%	21.1%	25.2%

Table 2: Objective-value agreement across datasets. The No-interact. row solves from the translator’s initial literals without questions. Deltas are percentage-point differences. “Exact” denotes error below

0.01\%

; “Gap” is mean objective error capped at

100\%

per instance; “Resolved” is

(\textsc{Known}+\textsc{Filled})/\text{all extracted literals}

Configuration	NL4LP	EasyLP	NL4OPT	ReSoc	IndOR	CmplxOR	CmplxLP	Macro avg.
SAILOR	87.6	79.8	78.1	77.9	64.3	50.0	27.0	66.4
w/o importance	89.9 $\uparrow$ 2.3	81.3 $\uparrow$ 1.5	77.6 $\downarrow$ 0.5	76.7 $\downarrow$ 1.2	61.9 $\downarrow$ 2.4	44.4 $\downarrow$ 5.6	30.8 $\uparrow$ 3.8	66.1 $\downarrow$ 0.3
w/o refinement	86.5 $\downarrow$ 1.1	80.7 $\uparrow$ 0.9	76.8 $\downarrow$ 1.3	77.9 $\approx$ 0	59.5 $\downarrow$ 4.8	38.9 $\downarrow$ 11.1	29.4 $\uparrow$ 2.4	64.2 $\downarrow$ 2.2

Table 3: Ablation: exact objective-value agreement for SAILOR and two ablated variants. Deltas are percentage-point changes relative to SAILOR ; the final column is an unweighted macro-average.

Figure 3: Sensitivity of exact objective-value agreement to the prior range. Narrow and wide priors behave inconsistently across datasets; annotations give aggregate percentage-point differences from the default.

Figure 4: Sensitivity of exact objective-value agreement to question budgets

10

and

50

relative to the default budget

30

. Effects are mixed across datasets.

Figure 5: Objective-value agreement for two translation models. Solid lines show exact agreement; shaded bands extend to agreement within

10\%

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Our masking pipeline. We take the original description, hide the decision-critical numbers, and optionally paraphrase the result so the underspecified version reads naturally.

Large Language Model-Guided Optimization

Large Language Model-Guided Optimization uses language models to propose, steer, and refine solutions to hard search and design problems.

See all 587 papers

OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale
Jul 29, 2024Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg +3Mixed-Integer Linear Programming Large Language Model-Guided Optimization
a School of Management Science and Engineering, Stanford University, Stanford, California 94305 · b Institute for Computational and Mathematical Engineering, Stanford University, Stanford, California 94305
ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modeling
May 4, 2026Guangrui XieMulti-Agent LLM Systems Large Language Model-Guided Optimization
Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization
Sep 4, 2026Sihan Ge, Yichen Lin, Chenyu Zhou +3Large Language Model-Guided Optimization LLM Agent Evaluation
Cardinal Operations · Shanghai Jiao Tong University, Shanghai, China

Jul 29, 2024J/K move · Enter open · S save

OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg, Shayan Talaei+2

a School of Management Science and Engineering, Stanford University, Stanford, California 94305 · b Institute for Computational and Mathematical Engineering, Stanford University, Stanford, California 94305

Mixed-Integer Linear Programming Large Language Model-Guided Optimization LLM-Based Program Synthesis Linear Programming Natural Language to Optimization Modeling Integer Linear Programming

Open paper AskPDF arXiv

Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare. However, most such problems are still solved heuristically by hand rather than optimally by state-of-the-art solvers because the expertise required to formulate and solve these problems limits the widespread adoption of optimization tools and techniques. We introduce a Large Language Model (LLM)-based system designed to formulate and solve (mixed integer) linear programming problems from their natural language descriptions. Our system can develop mathematical models, write and debug solver code, evaluate the generated solutions, and improve efficiency and correctness of its model and code based on these evaluations. OptiMUS is designed as a productivity tool for optimization practitioners who understand the problem domain and can describe it precisely, but seek to accelerate the modeling and implementation workflow. OptiMUS-0.3 utilizes a modular structure to process problems, allowing it to handle problems with long descriptions and complex data without long prompts. Experiments demonstrate that OptiMUS-0.3 outperforms direct-prompting baselines by over 43% on easy and 18% on hard instances. It remains competitive with fine-tuned specialist models on benchmark problems, and outperforms them on real-world case studies (28.6% vs. 0%) where fine-tuned models fail to generalize. Ablation studies show that modular architecture with error correction is central to these gains. A key finding is that system architecture is a stronger driver of performance than model capability. Structured decomposition with targeted error correction enables weaker models to match stronger models under naive prompting, and remains competitive with fine-tuned specialist models without retraining costs.

Abstract

Large Language Model-Guided Optimization

Explore similar work

OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

ORPilot: A Production-Oriented Agentic LLM-for-OR Tool for Optimization Modeling

Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization