PEARL: Solver-in-the-Loop Interactive Optimization Modeling from Natural Language
Organizations: Peking University · Great Bay University · Huawei Technologies Co., Ltd
Abstract
Optimization modeling is the process of translating real-world decision problems, often described in natural language, into formal mathematical formulations and executable solver code. While recent advances in large language models have shown promise in automating this process, many approaches focus on one-shot generation or rely on predefined execution-and-repair workflows, without explicitly learning a policy for tool use and revision. This stands in sharp contrast to real-world optimization modeling, which is inherently interactive and proceeds through repeated solve-debug-revise cycles. We introduce PEARL, a system for multi-turn solver-interactive optimization modeling that uses Python execution and mathematical programming solvers inside this loop. Rather than relying on a fixed repair workflow, PEARL learns when to test partial models, how to revise from solver diagnostics, and when to stop. It operates in a multi-turn tool-integrated setting where intermediate execution results, feasibility signals, and solution checks are used to improve both formulations and solver code before finalization. Across diverse optimization benchmarks, PEARL substantially improves executable objective agreement over the same-scale base model and is competitive with strong one-shot and tool-augmented baselines. Notably, PEARL-Qwen3-4B outperforms the much larger DeepSeek-V3.2-685B baseline under direct prompting on both macro- and micro-averaged Pass@1. Code and datasets are provided at https://github.com/AuroraLHL/PEARL.
Figures & tables
| Method | NL4Opt | MAMO-E | MAMO-C | NLP4LP | OptMATH | IndustryOR | ComplexOR | Resocratic | Macro Avg | Micro Avg | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Direct Prompt | Qwen-3-4B-Instruct | 48.13 1.40 | 2.20 0.45 | 13.51 2.20 | 55.62 1.25 | 2.41 0.50 | 16.66 3.10 | 5.56 2.78 | 33.50 1.80 | 22.20 0.75 | 22.42 0.55 |
| Qwen-3-4B-Think | 48.60 2.80 | 27.34 2.50 | 6.31 1.90 | 59.55 2.10 | 1.81 0.45 | 30.95 4.10 | 22.22 5.56 | 50.12 2.50 | 30.86 0.95 | 35.06 0.80 | |
| Qwen3-235B-A22B-Instruct-2507 | 74.92 1.77 | 91.62 0.69 | 38.74 5.41 | 88.95 0.33 | 20.88 3.09 | 58.73 5.50 | 38.89 0.00 | 70.64 1.41 | 60.42 1.13 | 72.26 0.69 | |
| Qwen3-235B-A22B-Think-2507 | 79.59 5.42 | 89.79 3.07 | 40.84 6.64 | 91.20 5.54 | 30.72 1.59 | 71.43 4.12 | 50.00 5.56 | 74.36 4.50 | 65.99 1.68 | 74.95 1.39 | |
| DeepSeek-V3.2-685B | 83.18 3.07 | 94.07 2.29 | 50.75 9.38 | 92.70 2.81 | 25.10 3.03 | 58.73 11.98 | 51.85 6.42 | 75.68 2.52 | 66.51 1.88 | 77.07 1.22 | |
| DeepSeek-V3.2-Think-685B | 85.20 1.18 | 95.53 0.59 | 53.15 3.12 | 97.38 0.86 | 30.32 1.25 | 64.29 2.39 | 50.00 5.56 | 77.67 0.43 | 69.19 0.65 | 79.58 0.41 |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Learning target | Role of solver feedback |
|---|---|---|
| SIRL | Outcome-based learning of complete generations | Terminal execution, feasibility, and objective-based scoring. |
| OR-R1 | Output-level RL | Solver outcomes score generated solutions. |
| MURKA | GRPO-trained Extractor and distilled Solver | A separate Checker iteratively verifies solutions through solver feedback. |
| StepORLM | Generative process supervision and preference learning | Solver outcomes and process feedback guide reasoning-trajectory learning. |
| STORM | Curriculum-guided output learning | Outcome signals support curriculum-shaped alignment. |
| PEARL | Multi-turn policy for execution, revision, and termination | Solver output conditions later actions; trajectory rewards combine format and objective agreement. |
| Hyperparameter | Value |
|---|---|
| Base model | Qwen3-4B-Instruct-2507 |
| Hardware | 1 node, 4 high-performance GPUs, Ray-based training |
| Rollout engine | SGLang tool-call rollout (colocated), 4 GPUs per engine |
| Algorithm | GRPO with decoupled clipping ( , ) |
| KL regularization | disabled in main runs, coefficient |
| Entropy regularization | coefficient |
| Parameter | Value |
|---|---|
| Provider | vLLM (optrl_vllm) |
| Model | Qwen/Qwen3-4B-Instruct-2507 |
| Tokenizer | Qwen/Qwen3-4B-Instruct-2507 |
| Max model length | 16384 tokens |
| Max output tokens | 16384 tokens |
| LLM configuration timeout | 300 seconds (see note below) |
| Method | NL4Opt | MAMO-E | MAMO-C | NLP4LP | OptMATH* | IndustryOR | ComplexOR | Resocratic | Macro Avg | Micro Avg |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen-3-8B | 38.32 1.60 | 38.53 2.20 | 12.61 2.30 | 46.07 1.90 | 3.01 0.55 | 2.38 1.35 | 5.56 2.78 | 37.47 1.70 | 22.99 0.70 | 32.56 0.60 |
| PEARL-Qwen-3-8B | 86.45 1.10 | 94.68 0.75 | 57.66 3.40 | 94.38 1.20 | 24.10 1.65 | 64.29 6.20 | 38.89 4.80 | 81.39 0.90 | 67.73 0.55 | 79.61 0.30 |
| PEARL-Qwen-3-4B-Instruct-2507 | 85.28 0.99 | 91.65 0.65 | 76.12 3.19 | 93.25 1.59 | 16.87 1.70 | 67.86 8.42 | 41.66 3.92 | 84.99 0.52 | 69.71 0.13 | 79.84 0.25 |
| Method | NL4Opt | MAMO-E | MAMO-C | NLP4LP | OptMATH | IndustryOR | ComplexOR | Resocratic | Macro Avg | Micro Avg |
|---|---|---|---|---|---|---|---|---|---|---|
| SFT-Qwen-3-4B-Instruct | 83.64 | 92.66 | 55.86 | 93.82 | 14.46 | 52.38 | 44.44 | 79.90 | 64.64 | 76.86 |
| PEARL-SFT-Qwen-3-4B-Instruct | 85.98 | 91.19 | 76.57 | 94.38 | 23.49 | 61.90 | 38.89 | 84.62 | 69.63 | 80.32 |
| PEARL-Qwen-3-4B-Instruct | 86.45 | 91.01 | 63.96 | 94.93 | 30.12 | 61.90 | 50 | 84.86 | 70.40 | 80.38 |
| Benchmark | Acc. | Tool Calls | ||
|---|---|---|---|---|
| Base | PEARL | Base | PEARL | |
| NL4Opt | 68.22 | 85.28 | 0.47 | 1.31 |
| MAMO-Easy | 41.28 | 91.65 | 0.85 | 1.42 |
| MAMO-Complex | 12.61 | 76.12 | 0.15 | 1.55 |
| NLP4LP | 81.29 | 93.25 | 0.19 | 1.29 |
| OptMATH | 5.42 | 16.87 | 0.23 | 4.43 |
| Problem pool | Approx. count | Sources and preparation |
|---|---|---|
| Cleaned public data | 10k | OR-Instruct/ORLM, OptMATH, and OptiBench-style resources; cleaning and reference-objective verification. |
| Expert-curated data | 3.8k | Public OR articles and websites; LLM-assisted description rewriting, expert formulation checks, filtering, and solver verification. |
| Benchmark | Instances | Released PEARL file |
|---|---|---|
| NL4Opt | 214 | nl4opt.jsonl |
| MAMO EasyLP | 545 | mamo_easy_lp_clean.jsonl |
| MAMO ComplexLP | 111 | mamo_complex.jsonl |
| NLP4LP | 178 | nl4lp.jsonl |
| OptMATH-Bench | 166 | OptMATH_Bench.jsonl |
| IndustryOR | 42 | IndustryOR.jsonl |
| Benchmark | PEARL | Official SIRL | Retrained SIRL | |
|---|---|---|---|---|
| Pyomo | Gurobi | Pyomo | ||
| NL4Opt | 214 | 90.19 | 87.85 | 84.51 |
| MAMO-E | 545 | 94.13 | 96.15 | 94.50 |
| MAMO-C | 111 | 87.39 | 79.28 | 44.14 |
| NLP4LP | 178 | 91.01 | 94.38 | 89.33 |
| OptMATH-Bench | 166 | 3.61 | 27.71 | 1.20 |
| Training problem pool | Macro | Micro |
|---|---|---|
| Cleaned public 10k | 64.12 | 78.95 |
| Curated 3.8k | 67.61 | 79.23 |
| Combined 13.8k | 70.40 | 80.38 |
| Method | Mean tokens | Execution success (%) | Pass@1 (%) |
|---|---|---|---|
| PEARL-Qwen3-4B | 9,868.75 | 90.88 | 79.43 |
| DeepSeek-V3.2 | 1,465.51 | 91.29 | 80.68 |
| DeepSeek-V3.2-Think | 2,323.56 | 91.06 | 79.90 |
| Benchmark | Pass@1 (%) |
|---|---|
| NL4Opt | 4.21 |
| MAMO-E | 8.62 |
| MAMO-C | 7.21 |
| NLP4LP | 5.62 |
| OptMATH-Bench | 0.00 |
| IndustryOR | 4.76 |
Natural Language to Optimization Modeling
Natural language to optimization modeling translates human requirements into formal mathematical models and solver-ready code.
See all 82 papers