OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
Organizations: University of Washington · Stanford University · Microsoft Research
Abstract
Mathematical optimization drives decisions in supply chains, logistics, energy, and scheduling, but translating natural-language problems into solver-executable formulations remains a bottleneck. While frontier models show promise for this task, many deployments require small language models (SLMs) that run locally under privacy, latency, and cost constraints. Existing SLMs are limited by noisy data, ambiguous benchmarks, and one-shot inference that ignores the iterative, feedback-driven nature of optimization modeling. To address these challenges, we introduce OptiMind, a system that combines data cleaning and class-specific hints with training using solver feedback. OptiMind distills recurring formulation errors into class-specific hints that guide the repair of ambiguous problems and the regeneration of solution trajectories. This process provides reliable training data, which is then used for multi-turn RL to teach models to revise their formulations in response to solver feedback. At inference time, the same hint library guides further refinement as models revise their solutions over successive turns. To support reliable evaluation, we clean three challenging benchmarks from the literature. On these cleaned benchmarks, our 20B OptiMind-RL model outperforms strong open-source baselines and reaches frontier-model performance under the same protocol. The full hint library, cleaned test data, cleaning logs, and evaluation code are publicly available on Github (https://github.com/microsoft/OptiGuide/tree/main/optimind), and our trained OptiMind-RL model is available on Hugging Face (https://huggingface.co/microsoft/OptiMind-RL).
Figures & tables
| IndustryOR | Mamo-Complex | OptMATH | |||||
|---|---|---|---|---|---|---|---|
| Model | Hints | 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns |
| GPT-OSS-20B | None | 66.23 | 71.89 | 65.03 | 86.80 | 66.68 | 81.52 |
| SFT | None | 68.97 | 72.43 | 85.77 | 90.71 | 72.95 | 81.64 |
| Multi-turn RL | None | 71.97 | 75.88 | 85.36 | 89.58 | 75.70 | 82.92 |
| SFT | Expert | 70.50 | 75.25 | 89.09 | 92.04 | 79.93 | 88.50 |
| Multi-turn RL | Expert | 75.76 | 78.54 | 90.00 | 92.26 | 85.17 | 90.99 |
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Core methodology | Policy optimization | Inference control |
|---|---|---|---|
| ORLM [ Tang et al., 2024 ] | OR-Instruct synthetic data and SFT | – | – |
| OptMATH [ Lu et al., 2025 ] | Bidirectional data synthesis for optimization modeling | – | – |
| LLMOPT [ Jiang et al., 2025 ] | Multi-instruction tuning and self-correction | – | Self-correction |
| SIRL [ Chen et al., 2025 ] | Solver-informed RL with executable code/model verification | RLVR | Instance-enhanced self-consistency |
| StepORLM [ Zhou et al., 2025 ] | Solver outcome verification plus generative process supervision | W-DPO | GenPRM verification/reranking |
| Solver-in-the-Loop [ Ao et al., 2026 ] | MDP-based OR model debugging with IIS/solver actions | GRPO | MDP repair policy |
| IndustryOR | Mamo-Complex | OptMATH | |||||
|---|---|---|---|---|---|---|---|
| Model | Hints | 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns |
| gpt-5 | no hint | ||||||
| expert | |||||||
| automatic | |||||||
| hybrid | |||||||
| o4-mini | no hint | ||||||
| IndustryOR-SIRL | IndustryOR-LogiOR | IndustryOR-Survey | Mamo-Complex-SIRL | Mamo-Complex-Survey | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Hints | 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns |
| gpt-5 | no | 69.67 | 71 | 68.29 | 70.12 | 66.67 | 68.25 | 72.57 | 74.05 | 65.76 | 66.21 |
| expert | 72 | 73 | 76.82 | 78.04 | 70.23 | 69.04 | 88.17 | 89.65 | 85.58 | 86.48 | |
| o4-mini | no | 61.33 | 64.67 | 66.26 | 74.79 | 68.25 | 69.84 | 57.96 | 69.95 | 58.55 | 66.67 |
| expert | 68.67 | 70 | 71.54 | 74.39 | 66.67 | 63.49 | 81.77 | 86.53 | 78.82 | 86.48 | |
| SIRL-Gurobi32B | no | 29.25 | 29.87 | 38.9 | 39 | 47.85 | 47.85 | 53.05 | 53.1 | 83.06 | 82.97 |
| IndustryOR | Mamo-Complex | OptMATH | |||||
|---|---|---|---|---|---|---|---|
| Model | Hints | 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns |
| Qwen3.5-9B | None | ||||||
| Expert | |||||||
| Qwen-SFT | None | ||||||
| Expert | |||||||
| Qwen-RL-multi | None | ||||||
| Model | Hints | 1 turn | 5 turns |
|---|---|---|---|
| GPT-OSS-20B | None | ||
| GPT-OSS-20B | Reused expert | ||
| OptiMind-SFT | None | ||
| OptiMind-SFT | Reused expert | ||
| OptiMind-RL-single | None | ||
| OptiMind-RL-single | Reused expert |
| Model | 1 turn | 5 turns |
|---|---|---|
| GPT-OSS-20B | ||
| OptiMind-SFT | ||
| OptiMind-RL-single | ||
| OptiMind-RL-multi | ||
| GPT-5 | ||
| Qwen3.5-9B |
| Model | Optimization | Agentic | LM-eval |
|---|---|---|---|
| GPT-OSS-20B | 65.79 | 64.75 | 56.50 |
| OptiMind-SFT | 78.19 | 0.00 | 48.28 |
| OptiMind-RL | 79.48 | 66.80 | 56.11 |
| RL variant | Hint strategy | IndustryOR | Mamo-Complex | OptMATH | |||
|---|---|---|---|---|---|---|---|
| 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns | ||
| Single-turn RL | No hint | ||||||
| Expert | |||||||
| Hybrid | |||||||
| Terminal reward only | No hint | ||||||
| Expert | |||||||
| Classification / Hint Strategy | 1 turn | 5 turn |
|---|---|---|
| gpt-oss classifier (ours) | 83.81 | 89.90 |
| Human expert classification | 81.04 | 89.61 |
| All hints, no classification | 78.57 | 89.23 |
| Random classes | 72.57 | 87.23 |
| No hints | 70.00 | 86.57 |
| Hint | gpt-oss-20B | OptiMind-SFT | ||||
|---|---|---|---|---|---|---|
| IndustryOR | Mamo-Complex | OptMATH | IndustryOR | Mamo-Complex | OptMATH | |
| Raw GPT-5 hints | 62.19 | 66.57 | 75.91 | 66.86 | 80.95 | 83.43 |
| Refined hints | 63.83 | 67.85 | 76.34 | 67.47 | 82.22 | 85.54 |
| Refined + filtered hints | 61.31 | 68.14 | 77.03 | 69.39 | 82.76 | 87.11 |
| Problem Class | Common error pattern | Hint |
|---|---|---|
| Traveling Salesman | Apply MTZ subtour-elimination constraints to the start city, making the ordering constraints infeasible. | Fix the start city’s position, and apply the MTZ constraint for each visited pair of cities except the start city. |
| Job Shop | Use two binaries for a pair of operations without exclusivity, e.g., both “ before ” and “ before ” allows inconsistent ordering. Big-M is too small, making the disjunction invalid and incorrectly ruling out feasible schedules. | Use one ordering binary per unordered pair: means operation is scheduled before . Set to plus the sum of processing times of all operations. |
| Capacitated Facility Lot-Sizing | Backlog is often modeled with the wrong sign on or by using a “net inventory” term ( ) inconsistently, which breaks flow conservation. | Use the exact balance equation . |
| Error pattern | Hint |
|---|---|
| Self-loop arcs are mishandled, either by adding undefined self-arcs or by forbidding self-arcs that represent local use, holding, or storage. | Build the arc set from the input data; exclude self-loops only when the instance has no explicit arcs, and otherwise preserve them or replace them with explicit local-use variables and matching cost/capacity constraints. |
| Missing node-wise flow conservation allows flow to be created or destroyed when the model only constrains sources and sinks. | Impose one flow-balance constraint at each node and commodity, e.g., outgoing flow minus incoming flow equals net supply, while adding separate restrictions only when the problem explicitly forbids inflow to sources or outflow from sinks. |
| IndustryOR | Mamo-Complex | OptMATH | ||||
|---|---|---|---|---|---|---|
| Hint strategy | 1 turn | 5 turns | 1 turn | 5 turns | 1 turn | 5 turns |
| No hint | ||||||
| Expert | ||||||
| Automatic ( gpt-oss-120b ) | ||||||
| Hybrid (expert + gpt-oss-120b ) | ||||||
| Automatic ( GPT-5 ) | ||||||
| Benchmark | Problem Index | Optimization Class | Why the original problem is incorrect | How we fixed it |
|---|---|---|---|---|
| IndustryOR | 24 | – | The problem did not state an objective function. | We added a well-specified objective (minimize pollution). |
| IndustryOR | 46 | – | The original instance was infeasible but had a ground-truth objective value of . | We updated the data to make the instance feasible and adjusted the ground-truth objective accordingly. |
| OptMATH | 35 | Facility Location Problem (FLP) | The problem describes selecting towers to cover regions, but the description is missing data about which towers cover which regions. | We generated the missing coverage data semi-automatically and then verified it manually. |
| OptMATH | 36 | Job Shop | The instance omits the number of machines and specifies only a partial assignment of jobs to machines. | We generated the missing machine and assignment data semi-automatically and then verified it manually. |
| Benchmark | # (%) Incorrect | Missing Data | Ambiguous | Infeasible | Wrong GT | Underspecified | Specific Solver | Non-linear | Inconsistent | Integral / Fractional |
|---|---|---|---|---|---|---|---|---|---|---|
| IndustryOR | 50 (50%) | 0 (0%) | 11 (11%) | 2 (2%) | 25 (25%) | 5 (5%) | 5 (5%) | 1 (1%) | 2 (2%) | 4 (4%) |
| MAMO-Complex | 106 (51.2%) | 15 (7.1%) | 3 (1.4%) | 2 (0.9%) | 42 (19.9%) | 0 (0%) | 0 (0%) | 0 (0%) | 4 (1.9%) | 43 (20.4%) |
| OptMATH | 68 (41%) | 48 (29%) | 11 (7%) | 1 (0.6%) | 10 (6%) | 1 (0.6%) | 0 (0%) | 0 (0%) | 0 (0%) | 0 (0%) |
| Model | Hints | 1 turn | 5 turns |
|---|---|---|---|
| GPT-OSS-20B | None | ||
| GPT-OSS-20B | Nonlinear | ||
| OptiMind-SFT | None | ||
| OptiMind-SFT | Nonlinear | ||
| OptiMind-RL-single | None | 24.24 | 43.18 |
| OptiMind-RL-single | Nonlinear | 86.74 | 89.39 |
| Parameter | Value |
|---|---|
| Batch size | 16 |
| Learning rate | 2e-5 |
| Max prompt length | 16384 |
| Max response length | 16384 |
| Prompt length truncation | left |
| Max epochs | 3 |
| Parameter | Value |
|---|---|
| Algorithm | GRPO |
| Initialization | gpt-oss-20b-bf16 |
| Rollout turns | 3 |
| GRPO group size | 2 |
| Samples per prompt | 8 |
| Prompts per step | 32 |
| Parameter | Value |
|---|---|
| Batch size | 32 |
| Learning rate | 1e-5 |
| Max sequence length | 32768 |
| Sequence length truncation | left |
| Max epochs | 3 |
| Parameter | Value |
|---|---|
| Algorithm | GRPO |
| Initialization / precision | Qwen3.5-9B / BF16 |
| Rollout turns | 3 (early stopping disabled) |
| Samples per prompt | 8 |
| Prompts per rollout | 32 |
| Global / micro batch size | 256 / 1 |
| Parameter | Value |
|---|---|
| Temperature | 0.9 |
| Top p | 0.95 |
| Max tokens | 60000 |
| Aircraft Assignment | Aircraft Landing | Bin Packing |
| Blending Problem | Capacitated Facility Location Problem | Capacitated Lot-sizing Problem (CLSP) |
| Car Selection Assignment | Contract Allocation | Diet Problem |
| Factory Planning Problem | Flow Shop Scheduling | Job Shop |
| Knapsack | Multicommodity Capacitated Network Design | MarketShare |
| Set Multi-Cover | PortfolioOptimization | Revenue Management Problem |
| Assignment Problem | Set Cover | Discrete Lot-sizing and Scheduling Problem |
| Subclass | Natural Language Example |
|---|---|
| Knapsack | A hiker is preparing for a 3-day outdoor hiking trip. They need to select the most valuable combination of equipment and supplies from many available options within the limited backpack capacity. The items include: * Tent: weight 5kg, value 90 points * Sleeping bag: weight 3kg, value 80 points* Portable stove: weight 1kg, value 60 points * Drinking water: weight 2kg, value 70 points * Dry food: weight 1.5kg, value 65 points * First aid kit: weight 0.5kg, value 50 points Assuming the backpack has a maximum weight capacity of 10kg, the hiker’s goal is to select the combination of items with the highest total value while not exceeding the weight limit. Each item must be either taken in its entirety or left behind; partial items cannot be taken. |
| Team Formulation Problem | As a team leader, you have several projects requiring different expertise levels in different areas. You want to assign a set of people to these projects. Each person has a level of expertise in each of the areas. How to assign people to projects to minimize the maximum skill shortage from the project expertise requirement? |
| Transportation Problem | A company has three warehouses (origins) with supplies of 100, 150, and 200 units, respectively. There are four retail stores (destinations) with demands of 80, 120, 90, and 160 units, respectively. The cost of transporting one unit from each warehouse to each retail store is given. The goal is to determine the optimal transportation plan that minimizes the total transportation cost while meeting all supply and demand constraints. |
Natural Language to Optimization Modeling
Natural language to optimization modeling translates human requirements into formal mathematical models and solver-ready code.
See all 82 papers