Log-Driven Semantic Tuning for Mixed Integer Programming Solvers
Organizations: Alibaba Group US, Bellevue, WA, USA
Abstract
Automated configuration of mixed-integer programming (MIP) solvers largely relies on black-box optimization. Each trial requires an expensive solver run and provides only a scalar outcome, so tuning can require many such evaluations. Human experts take a different approach. They inspect solver logs, identify performance bottlenecks, and adjust the parameters that address them. We propose Log-Driven Semantic Tuning (LDST), which uses a large language model (LLM) to automate this process. LDST extracts compact diagnostics from solver logs, uses the LLM to propose sparse edits to the current best configuration based on parameter meanings and observed solver behavior, and validates each proposal before execution. A regime-adaptive objective selects the tuning signal according to the information available from a time-limited run, while a log-based completion signal estimates completion time from branch-and-bound tree growth recorded in ordinary solver logs, so that solved and unsolved runs share one time scale. On 215 instances from five benchmark sets, LDST reduces geometric-mean loss by 43.9% on easy instances and 16.7% on intermediate instances relative to the best of SMAC3, RBFOpt, and random search. In both regimes, LDST achieves a lower loss with 10 proposals than any of these baselines achieves with 50. Configurations that LDST tunes on a single instance also transfer better than baseline configurations to unseen instances of the same family.
Figures & tables
| Case | Level | Outcome | Signal | Summaries |
|---|---|---|---|---|
| Case 1 | Easy | solved | solve rate, time | |
| Case 2 | Intermediate | feasible, no proof | PDI | primal, dual, gap |
| Case 3 | Hard | dual progress only | norm. dual improv. | final dual, improved-dual coverage |
| Set | Source | # | Case 1 | Case 2 | Case 3 | Set | Source | # | Case 1 | Case 2 | Case 3 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| CVRP | Lima et al. (2014) | 90 | 11 | 39 | 40 | UFL | Hoefer (2018) | 30 | 30 | 0 | 0 |
| GAP | Beasley (1990) | 12 | 7 | 5 | 0 | MIPLIB | Gleixner et al. (2021) | 59 | 21 | 30 | 8 |
| JSP | Beasley (1990) | 24 | 7 | 17 | 0 | ||||||
| Total | 215 | 76 | 91 | 48 | |||||||
| Primary loss | Case 1 | Case 2 | |||||
|---|---|---|---|---|---|---|---|
| Method | Case 1 | Case 2 | Case 3 | Time (s) | Norm. primal | Norm. dual | Gap (%) |
| LDST | 0.274 | 0.557 | 0.993 | 10.5 | 1.010 | 0.990 | 4.9 |
| SMAC3 | 0.499 | 0.668 | 0.993 | 17.3 | 1.034 | 0.975 | 10.3 |
| RBFOpt | 0.490 | 0.675 | 0.988 | 17.1 | 1.039 | 0.975 | 10.8 |
| Random | 0.519 | 0.669 | 0.993 | 17.9 | 1.030 | 0.977 | 10.0 |
| Method | Scalar history | Parsed diagnostics | Parameter semantics | Best loss | (%) | |
|---|---|---|---|---|---|---|
| LDST | ✓ | ✓ | ✓ | 0.308 | 0.485 | 90.8 |
| LLM-StaticContext | ✗ | ✓ | ✓ | 0.342 | 0.509 | 86.8 |
| LLM-OutcomeOnly | ✓ | ✗ | ✓ | 0.358 | 0.520 | 84.2 |
| LLM-IndexOnly | ✓ | ✓ | ✗ | 0.523 | 0.707 | 76.3 |
| Overall | Family-level Case 1 test | ||||||
|---|---|---|---|---|---|---|---|
| Method | Case 1 train | Case 1 test | Test train | CVRP | GAP | JSP | UFL |
| LDST | 0.238 | 0.499 | 0.261 | 1.190 | 0.734 | 0.262 | 0.385 |
| SMAC3 | 0.429 | 0.799 | 0.370 | 2.067 | 0.966 | 0.657 | 0.565 |
| RBFOpt | 0.427 | 0.798 | 0.370 | 1.978 | 1.078 | 0.716 | 0.547 |
| Random | 0.449 | 0.838 | 0.390 | 1.866 | 1.313 | 0.635 | 0.601 |
| Case 1 train | Case 1 test | Test train | Case 2 | Case 3 | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Indep. | Joint | Indep. | Joint | Indep. | Joint | Indep. | Joint | Indep. | Joint | |
| 0.390 | 0.282 | 0.540 | 0.508 | 0.150 | 0.226 | 0.919 | 0.849 | 1.010 | 1.002 | |
| 0.396 | 0.325 | 0.495 | 0.468 | 0.099 | 0.143 | 0.864 | 0.828 | 1.018 | 1.011 | |
| 0.421 | 0.395 | 0.560 | 0.561 | 0.139 | 0.166 | 0.715 | 0.689 | 1.025 | 1.007 | |
| 0.389 | 0.435 | 0.543 | 0.625 | 0.154 | 0.190 | 0.687 | 0.775 | 1.030 | 1.008 | |
| 0.467 | 0.376 | 0.474 | 0.508 | 0.007 | 0.132 | 0.705 | 0.720 | 1.009 | 1.008 | |
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
| Family | Source collection | Construction route | # | Case 1 | Case 2 | Case 3 |
|---|---|---|---|---|---|---|
| CVRP | CVRPLIB | two-commodity flow MILP export | 90 | 11 | 39 | 40 |
| GAP | OR-Library gapc/gapd | assignment MILP export | 12 | 7 | 5 | 0 |
| JSP | OR-Library jobshop1 | disjunctive MILP export | 24 | 7 | 17 | 0 |
| UFL | UflLib Uniform | facility-location MILP export | 30 | 30 | 0 | 0 |
| MIPLIB | MIPLIB 2017 benchmark-v2 | direct MPS benchmark instances | 59 | 21 | 30 | 8 |
| Family | Instance | Case | Family | Instance | Case |
|---|---|---|---|---|---|
| CVRP | A-n32-k5 | 2 | CVRP | A-n33-k5 | 2 |
| CVRP | A-n33-k6 | 2 | CVRP | A-n34-k5 | 2 |
| CVRP | A-n36-k5 | 2 | CVRP | A-n37-k5 | 2 |
| CVRP | A-n37-k6 | 2 | CVRP | A-n38-k5 | 2 |
| CVRP | A-n39-k5 | 2 | CVRP | A-n39-k6 | 2 |
| CVRP | A-n44-k6 | 2 | CVRP | A-n45-k6 | 3 |
| Family | Instance | Case | Family | Instance | Case |
|---|---|---|---|---|---|
| JSP | ft20 | 2 | JSP | la01 | 1 |
| JSP | la02 | 1 | JSP | la03 | 1 |
| JSP | la04 | 1 | JSP | la05 | 1 |
| JSP | la06 | 2 | JSP | la07 | 2 |
| JSP | la08 | 2 | JSP | la09 | 2 |
| JSP | la10 | 2 | JSP | la11 | 2 |
| Family | Case 1 pool | C2 train sizes | Splits per | Case 1 test count | Other transfer targets |
|---|---|---|---|---|---|
| CVRP | 11 | 5 | Case 2 and Case 3 | ||
| GAP | 7 | 5 | Case 2 only | ||
| JSP | 7 | 5 | Case 2 only | ||
| UFL | 30 | 5 | none beyond Case 1 |
| Component | Contract |
|---|---|
| Solver call | SCIP 9.2.4, one thread, one blocking solver run per evaluated configuration and instance, with per-run time limit . |
| Objective and normalization | The regime-adaptive loss in Section 3 is fixed before comparison and used identically by LDST, LLM controls, and scalar-feedback baselines; each method’s own default run (trial 0) defines its normalization reference. |
| Reported selection | For every method, the reported configuration after trials is the best-so-far configuration over trial 0 (the default) and proposals . |
| Evaluation budget | Each tuning trial consumes one sequential solver evaluation per active instance. Experiment-specific values of are stated in the corresponding experiment descriptions and split manifests. |
| Editable space | All methods search the same curated whitelist of 45 SCIP parameters in Appendix C ; numeric entries use the native linear scale. |
| LLM proposal contract | LLM-based methods use edit cap , edits around the current best configuration, the same JSON schema, the same re-query limit , and the same archive novelty check. |
| Method | Search-space encoding | Feedback | Settings and invalid handling |
|---|---|---|---|
| SMAC3 | Typed ConfigSpace over the whitelist; native linear numeric domains. | scalar loss | Deterministic, seed 42, sequential evaluations; non-finite values noncompetitive. |
| RBFOpt | Native float/int bounds; Boolean and categorical dimensions integer-encoded. | scalar loss | Seed 42, no local search, genetic global search, sequential evaluations; non-finite values noncompetitive. |
| Random | Independent typed sampling; positive real intervals log-uniform, all other numeric intervals uniform. | scalar loss | Seed 42, sequential samples, no surrogate; the default enters only through trial 0. |
| Method | Scalar hist. | Parsed diag. | Param. sem. | Form. text | Rationale | Validator |
|---|---|---|---|---|---|---|
| BBO baselines | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ |
| LLM-StaticContext | ✗ | ✓ | ✓ | optional | ✓ | ✓ |
| LLM-OutcomeOnly | ✓ | ✗ | ✓ | optional | ✓ | ✓ |
| LLM-IndexOnly | ✓ | ✓ | ✗ | optional | ✓ | ✓ |
| LDST | ✓ | ✓ | ✓ | optional | ✓ | ✓ |
| SCIP parameter path | Type | Default | Allowed domain | Scale |
|---|---|---|---|---|
| branching/scorefunc | cat | p | {s, p, q} | lin. |
| branching/preferbinary | bool | false | {false, true} | lin. |
| branching/relpscost/minreliable | float | 1 | [0, 1000000000] | lin. |
| branching/relpscost/maxreliable | float | 5 | [0, 1000000000] | lin. |
| branching/relpscost/sbiterquot | float | 0.5 | [0, 1000000000] | lin. |
| branching/relpscost/sbiterofs | int | 100000 | [0, 1000000000] | lin. |
| Item | Value |
|---|---|
| Best and recent records in prompt memory | 3, 3 |
| LLM model | GPT-5.2 |
| Reasoning effort | medium |
| Max output tokens | 20,000 |
| Block | Parser fields exposed to the policy | Implementation definition | SCIP source |
|---|---|---|---|
| Run size and status | status , t_total (shown as solve_time ), nodes_last (shown as solving_nodes ), nodes_left , max_depth | t_total is the reported solving time, or if missing; nodes_last is the reported solving-node count, or if missing. nodes_left and max_depth are the B&B-tree summary values. | final summary, B&B tree, progress rows |
| Bounds and integral | primal_final , dual_final , gap_final , pdi (shown as primal_dual_integral ) | primal_final and dual_final are final-summary bounds, falling back to and . gap_final . pdi is the SCIP-reported primal–dual integral parsed from the Integrals block. | final summary, Integrals block |
| Incumbents | nsols , num_improvements , first_inc_time (shown as first_incumbent_time ), first_primal | nsols and num_improvements are solution-summary counts. first_inc_time and first_primal use the first-solution summary when present; otherwise they use the time and value at . | solution summary, progress rows |
| Presolve | presolve_time , presolved_vars , presolved_cons_init , presolved_cons_max | Direct parser entries for presolving time, presolved-variable count, initial presolved-constraint count, and maximum presolved-constraint count. | presolve summary |
| Root progress | root_dual0 , root_dual_end , root_dual_gain , root_gain_rel , root_first_lp_iters , root_first_lp_time , root_final_root_iters , root_lp_estimate | root_dual0 is the first root-LP value and root_dual_end is the final root dual bound. The derived fields are root_dual_gain and root_gain_rel . | root-node report |
| LP and branching effort | sb_time , sb_calls , sb_iters , sb_root_calls , sb_root_iters | sb_time , sb_calls , and sb_iters are parsed from the LP-statistics row for strong branching; sb_root_calls and sb_root_iters are parsed from the root-node LP row. | LP statistics |
| Prompt block in Figure 5 | LDST | StaticContext | OutcomeOnly | IndexOnly |
|---|---|---|---|---|
| Header, whitelist, current best, edit cap, schema | semantic names, defaults, domains | same | same | indices and domains |
| Optional formulation text | optional | optional | optional | optional |
| Validator feedback during re-query | kept | re-query only | kept | kept |
| [MEMORY] archive summary | scalar + diagnostics | current diagnostics only | scalar only | full, indexed |
| Multi-instance diagnostic rows | kept | removed | removed | kept |
| [TIPS] diagnostic-feature guide | kept | removed | removed | kept |
| acc. | SMAPE | mean | med | med. fit (s) | |||
|---|---|---|---|---|---|---|---|
| 0.50 | 100 | 0.88 | 0.149 | 0.879 | 0.889 | 0.909 | 0.104 |
| 0.60 | 100 | 0.92 | 0.145 | 0.981 | 0.997 | 1.026 | 0.123 |
| 0.70 | 100 | 0.97 | 0.145 | 1.028 | 1.044 | 1.040 | 0.122 |
| 0.80 | 100 | 0.99 | 0.110 | 1.022 | 1.035 | 1.009 | 0.115 |
| 0.90 | 100 | 0.99 | 0.068 | 1.017 | 1.024 | 1.003 | 0.102 |
| Signal at the cutoff | Correctly ordered pairs | Mean within-instance |
|---|---|---|
| Completion estimate | 26/34 | 0.515 |
| Final logged gap | 25/34 | 0.485 |
| Logged-gap integral (PDI proxy) | 17/34 | 0.121 |
| Case | Proposals | Median | Mean | 90th pct. | Max | (%) | At cap (%) |
|---|---|---|---|---|---|---|---|
| Case 1 | 5 | 5.00 | 8 | 19 | 97.8 | 0.0 | |
| Case 2 | 7 | 7.35 | 12 | 20 | 86.5 | 0.4 | |
| Case 3 | 11 | 11.71 | 18 | 20 | 49.3 | 4.7 | |
| All | 7 | 7.49 | 13 | 20 | 82.2 | 1.2 |
| Benchmark | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Indep. | Joint | Indep. | Joint | Indep. | Joint | Indep. | Joint | Indep. | Joint | |
| CVRP | 1.122 | 0.796 | 1.014 | 0.874 | 0.985 | 0.885 | 1.028 | 0.877 | 0.769 | 0.675 |
| GAP | 0.775 | 0.673 | 0.841 | 0.680 | – | – | – | – | – | – |
| JSP | 0.265 | 0.325 | 0.217 | 0.204 | – | – | – | – | – | – |
| UFL | 0.369 | 0.382 | 0.323 | 0.395 | 0.318 | 0.355 | 0.287 | 0.445 | 0.292 | 0.382 |
| Benchmark | Indep. C1 | Joint C1 | Indep. C2 | Joint C2 | Indep. C3 | Joint C3 |
|---|---|---|---|---|---|---|
| CVRP | 1.014 | 0.874 | 0.776 | 0.749 | 1.018 | 1.011 |
| GAP | 0.841 | 0.680 | 1.030 | 0.913 | – | – |
| JSP | 0.217 | 0.204 | 0.807 | 0.830 | – | – |
| UFL | 0.323 | 0.395 | – | – | – | – |
| Instance | LDST | SMAC3 | RBFOpt | Random | |
|---|---|---|---|---|---|
| miplib_irish-electricity | 0.9533 | 0.9218 | 0.6977 | 0.8991 | |
| miplib_neos-1354092 | 0.9215 | 0.9863 | 1.0000 | 1.0000 | |
| cvrp_B-n51-k7 | 0.9922 | 0.9804 | 0.9855 | 0.9860 | |
| cvrp_B-n45-k6 | 0.9942 | 0.9893 | 0.9894 | 0.9895 | |
| cvrp_A-n62-k8 | 0.9964 | 0.9918 | 0.9919 | 0.9918 | |
| cvrp_P-n50-k10 | 0.9919 | 0.9959 | 0.9960 | 0.9960 |
| Item | Value |
|---|---|
| Operating system | Linux (Ubuntu; kernel 6.14.0-33-generic; x86_64) |
| CPU | 13th Gen Intel(R) Core(TM) i7-13700K (24 logical CPUs) |
| System memory | 62 GiB total RAM |
| Python | 3.12.4 |
| SCIP | 9.2.4 |
| Threads per solver run | 1 |
| Set | # | Range of | Source |
|---|---|---|---|
| A | 27 | 32–80 | Augerat et al. (1995) |
| B | 23 | 31–78 | Augerat et al. (1995) |
| E | 13 | 13–101 | Christofides and Eilon (1969) |
| F | 3 | 45–135 | Fisher (1994) |
| P | 24 | 16–101 | Augerat et al. (1995) |
| Total | 90 |
| File | # | Sizes | Source |
|---|---|---|---|
| gapc | 6 | Chu and Beasley (1997) | |
| gapd | 6 | ||
| Total | 12 |
| Family | # | Range of | Source |
|---|---|---|---|
| abz(5-9) | 5 | – – | Adams et al. (1988) |
| ft(10,20) | 2 | – – | Fisher and Thompson (1963) |
| la(01-12) | 12 | – | Lawrence (1984) |
| orb(01-05) | 5 | Applegate and Cook (1991) | |
| Total | 24 |
| Family | # | Source | |
|---|---|---|---|
| *UnifS | 30 | Hoefer (2018) | |
| Total | 30 |
| Final selected instances ( benchmark-v2 ) | ||
|---|---|---|
| 50v-10 | assign1-5-8 | atlanta-ip |
| b1c1s1 | beasleyC3 | cbs-cta |
| chromaticindex512-7 | cmflsp50-24-8-8 | CMS750_4 |
| cod105 | comp21-2idx | cryptanalysiskb128n5obj16 |
| dano3_3 | drayage-25-23 | dws008-01 |
| fhnw-binpack4-48 | gfd-schedulen180f7d50m30k18 | glass4 |
Hyperparameter Optimization
Hyperparameter optimization selects model and training settings to improve predictive performance, efficiency, or reliability under limited evaluation budgets.
See all 178 papers