Agentic AI-Assisted Modeling for Production Scheduling: Assessment in Constraint Programming

Ángel Sánchez-Fernández; Javier Pernas-Álvarez; Diego Crespo-Pereira

cs.AIOct 7, 2026

Agentic AI-Assisted Modeling for Production Scheduling: Assessment in Constraint Programming

Authors: Ángel Sánchez-Fernández, Javier Pernas-Álvarez, Diego Crespo-Pereira

Organizations: Universidade da Coru˜na, Campus Industrial de Ferrol, CITENI, Grupo Integrado de Ingenier´ıa, Esteiro s/n, Ferrol, 15403.

Abstract

Developing optimization models for production scheduling requires substantial expert effort. Research on large language models (LLMs) has followed two directions: specialized approaches for automated modeling, mostly for mixed-integer linear programming, which often rely on dedicated training or problem-specific architectures that limit industrial deployment; and agentic artificial intelligence for operational decision support, which generally assumes that the optimization model already exists. This study bridges both directions by assessing whether general-purpose LLMs, orchestrated as agents without task-specific training, can formulate and implement constraint programming models from natural-language problem descriptions. Singleagent and multi-agent architectures are integrated with a Model Context Protocol server that provides context-aware retrieval of solver documentation to mitigate hallucinations during implementation. Both are compared with a direct LLM baseline on six industry-oriented problems covering flow-shop, job-shop, flexible job-shop and resource-constrained warehouse scheduling, using three LLMs and assessing modeling accuracy, execution success, latency and token consumption. Formulation proves largely within reach of current LLMs, whereas implementation is the main barrier. The multi-agent workflow raises the share of scripts that run correctly as generated from 14.8% with a direct LLM call to 59.3%, reaching 80.6% on the four less complex problems, while tightly coupled intralogistics models remain an open challenge.

Figures & tables

Work	Approach Summary	Method	Train	Scalability (Sol $\mid$ Prob $\mid$ LLM)
OptiMind	SLM fine-tuned and hint injection	MILP	$\checkmark$	$\ominus$ $\mid$ $\bullet$ $\mid$ $\circ$
OptiMUS-0.3	Modular agent with constraint graph & reflective correction	MILP	$\times$	$\bullet$ $\mid$ $\bullet$ $\mid$ $\bullet$
SMILO	Modelling graph-guided tasks & template assembly	MILP	$\times$	$\ominus$ $\mid$ $\ominus$ $\mid$ $\bullet$
Chen et al.	Domain specialization via knowledge-augmented fine-tuning	MILP	$\checkmark$	$\ominus$ $\mid$ $\circ$ $\mid$ $\circ$
El Baz et al.	Agentic model with human-in-the-loop OR expert	MILP	$\times$	$\ominus$ $\mid$ $\circ$ $\mid$ $\bullet$
RideAgent	Small-sample guided heuristic agent for variable fixing	MILP	$\times$	$\ominus$ $\mid$ $\circ$ $\mid$ $\bullet$

Table 1: Recent contributions for automated mathematical modelling with LLMs.

Workflow	Component	Task	MCP tools	Max. iterations a
(a) Multi-agent	Formulator Agent	Formulation and implementation plan	–	10
	Executor Agent	Coding of the given formulation	6	50
(b) Single agent	Executor Agent	Formulation and coding	6	50
(c) Direct LLM	LLM call	Formulation and coding	–	1

Table 2: Langflow components of each workflow. All components share the same LLM settings: temperature 0.15 and a limit of 15 000 output tokens per call.

Figure 1: Three workflows compared in this study. (a) Two-stage multi-agent pipeline: a tool-free Formulator Agent hands a formulation to an Executor Agent connected to the MCP documentation server. (b) Single Executor Agent connected to the same MCP server. (c) Direct LLM baseline without tool access.

Figure 2: Structure of the prompts. The task message that enters every workflow, shown for problem PC1: four sections in a fixed order, with the process described in natural language only, the identity of the SQLite instance, the schema of every table generated from the database file, and the conventions left open by the description. The complete prompts and the task messages of PC1–PC6 are provided as supplementary material and in the project repository ( Sánchez-Fernández et al., 2026 ) .

Term	Definition	Example
Module	A docplex.cp sub-package, documented in one reference file (four in the corpus)	docplex.cp.model
Symbol	A documented function, class or method (186, 15 and 132 in the corpus)	method solve of class CpoModel
Qualified name	Unique path of a symbol: module, class (if any) and name. Short names may be ambiguous	docplex.cp.model.CpoModel.solve short name solve ; add matches two classes
Chunk	Retrieval unit: the complete entry of one symbol, or one whole example script, stored under a deterministic identifier	api:docplex.cp.model: docplex.cp.model.CpoModel.solve ( $\approx$ 535 tokens) example:flow_shop_permutation

Table 3: Nomenclature of the documentation corpus, illustrated with the method solve of class CpoModel .

Figure 3: Layered architecture of the documentation MCP server. Token counts are estimated at four characters per token. Every tool response ends with a footer stating the next required step of the retrieval protocol.

Family	Tool	Input $\rightarrow$ Output a	Key features
Discovery capped	search_docs	[concepts] $\rightarrow$ candidate names	Merges all queries and groups hits by module; ends with the ready-made get_signature call
	list_functions	module $\rightarrow$ symbol index	Browsing fallback when keyword search finds nothing
	list_examples	problem text $\rightarrow$ ranked examples	Ranked by similarity to the description; alphabetical without one
Precision complete	get_signature	[names] $\rightarrow$ complete entries	Order kept; unknown names form a gap list, never approximated
	get_example	[ids] $\rightarrow$ full source	Complete working models, up to $\approx$ 5k tokens each
	get_module_reference	module (+ [names]) $\rightarrow$ reference	For many symbols of one module; $\approx$ 28k tokens for modeler

Table 4: Tools exposed by the MCP server. Discovery responses are non-authoritative and capped at 400 characters per summary line and 40 candidates per call; precision responses are authoritative and never truncated.

Figure 4: Retrieval protocol of the Executor Agent. Top: the four steps, the tools used at each step and the in-server enforcement. Bottom: example of the tool-call trace; calls 3–4 form a repair cycle triggered by the gap list returned in call 2.

Factor	Levels	Codes a	$n$
Workflow	(a) multi-agent + MCP; (b) single agent + MCP; (c) direct LLM (Figure 1 )	WA, WB, WC	3
LLM b	Gemini 2.5 Pro; GPT-5; DeepSeek V4 Pro	Gem, GPT, DS	3
Problem	PC1–PC6 (Table 6 )	PC1, …, PC6	6
Replicate	Independent generations	r1, r2, r3	3
Runs ( $3\times 3\times 6\times 3$ )			162

Table 5: Full-factorial experimental design and run nomenclature.

		PC1	PC2	PC3	PC4	PC5	PC6
Problem	Type a	FSP	JSP	FJSP	RCS	RCS	RCS
Sets	Jobs / orders	11 lots	10 orders	6 batches	29 orders	29 orders	20 orders
	Activities	55 operations	44 operations	18 operations	139 transports	348 transports b	360 transports b
	Resources	5 machines	6 work centres	8 machines, 3 operators	1 AGV, 1 station	1 AGV, 1 station	2 AGVs, 2 robots, buffer of 5
	Other	4 product families	–	38 eligible pairs	5 pallets, 5 kit types	12 pallets, 5 kit types	18 pallets, 5 kit types
Variables	Interval	55	44	18	168	29	20

Table 6: Benchmark problems: structure, size of the reference CP model and ground-truth result.

Dimension	Metric	Definition
Effectiveness	$\mathrm{Acc}_{\mathrm{F+C}}$ (%)	Percentage of replications of the same experimental setting for which the generated script executes successfully without modification and returns an objective value satisfying $z=z_{\mathrm{ref}}$
	$\mathrm{Acc}_{\mathrm{F}}$ (%)	Percentage of replications of the same experimental setting that reach $z_{\mathrm{ref}}$ after implementation errors are corrected, without modifying the underlying mathematical formulation
Efficiency	Tokens	Input and output tokens of all LLM calls of the run
	Time (s)	Wall-clock time of the complete workflow
	Tool calls	Number of calls to the MCP documentation server (workflows (a) and (b))
Diagnostics	Model outcome	Result of the generated model, as generated and after correction: reaches $z_{\mathrm{ref}}$ ; feasible but worse than $z_{\mathrm{ref}}$ ; invalid (objective below a proven bound); no solution within 60 s or proven infeasible; no objective reported; execution error; no complete script

Table 7: Evaluation metrics reported for each run.

			Accuracy (%)		Tokens (k)		Time	Tool
Workflow	Problem	LLM	F	F+C	Input	Output	(s)	calls
WA	PC1 (FSP)	Gem	100.0	100.0	152.1 $\pm$ 3.1	21.8 $\pm$ 1.5	202 $\pm$ 16	6.0 $\pm$ 0.0
		GPT	100.0	66.7	414.2 $\pm$ 173.7	24.4 $\pm$ 6.2	333 $\pm$ 84	10.3 $\pm$ 3.1
		DS	100.0	66.7	1108.6 $\pm$ 179.7	19.6 $\pm$ 3.2	516 $\pm$ 135	21.7 $\pm$ 2.5
	PC2 (JSP)	Gem	100.0	100.0	140.3 $\pm$ 25.0	20.1 $\pm$ 5.0	178 $\pm$ 42	6.3 $\pm$ 1.2
		GPT	100.0	100.0	606.7 $\pm$ 121.8	26.6 $\pm$ 4.0	372 $\pm$ 72	13.7 $\pm$ 2.3

Table 8: Summary of results per experimental setup (workflow, problem and LLM): accuracy at both levels, token consumption, inference time and MCP tool calls.

Panel of Figure 5 — Figure 5: Accuracy per problem and workflow across the three LLMs: (a) formulation + coding accuracy, $\mathrm{Acc}_{\mathrm{F+C}}$ ; (b) formulation accuracy, $\mathrm{Acc}_{\mathrm{F}}$ . Mean $\pm$ standard deviation over the nine runs of each workflow and problem (three LLMs $\times$ three replicates).

Panel of Figure 6 — Figure 6: Formulation + coding accuracy, $\mathrm{Acc}_{\mathrm{F+C}}$ , per problem, LLM and workflow: mean $\pm$ standard deviation over three replicates.

Panel of Figure 7 — Figure 7: Formulation accuracy, $\mathrm{Acc}_{\mathrm{F}}$ , per problem, LLM and workflow: mean $\pm$ standard deviation over three replicates.

Panel of Figure 8 — Figure 8: Inference time (bars, left axis) and tokens per run (diamonds, right axis, logarithmic scale) per LLM and workflow: mean $\pm$ standard deviation over the 18 runs of each workflow–LLM pair.

Script as generated (single execution, no edits)
	Workflow			Problem
Outcome of the generated model	(a)	(b)	(c)	PC1–PC4	PC5	PC6	All
Runs	54	54	54	108	27	27	162
Reaches $z_{\mathrm{ref}}$ (correct model)	32	18	8	53	5	–	58
Invalid: objective below a proven bound	–	–	1	–	1	–	1
No solution within 60 s or proven infeasible	7	5	2	–	4	10	14

Table 9: Outcome of the 162 generated models, as generated and after correcting their implementation errors, aggregated per workflow and per problem.

Figure 9: Error analysis of the generated scripts. (a) Outcome of the 54 scripts of each workflow: correct as generated, or first point at which the script fails; result reporting includes the 14 scripts that fail after printing $z_{\mathrm{ref}}$ , and the last class groups the scripts that run to completion without a valid solution. (b) The twelve most frequent implementation errors among the 159 distinct errors of the 104 incorrect runs, grouped by nature: hallucinated symbol (a name that does not exist in docplex.cp ), misuse of an existing element of the API, silent semantic error, and format, environment or generic Python defects.

Explore similar work

Aug 8, 2026cs.AI

Improving Constraint Models with LLM Agents

The runtime of Constraint Programming (CP) solvers is highly sensitive to modeling choices, such as symmetry breaking, implied constraints, global constraints, constraint reformulation, and variable representation. Improving these constraint models has traditionally required human expertise, and existing automated reformulation systems are restricted to a predefined library of hand-crafted transformation rules. We introduce an agentic framework that instead reformulates a constraint model from an open-ended space and establishes correctness empirically rather than by construction: a Large Language Model (LLM) agent, given a model and three training instances, proposes alternative formulations, validates each by injecting its solution back into the original model, and diagnoses and repairs failures, returning the best variant it finds in a median of about fifteen minutes. The models are expressed in the CPMpy modeling library, and each proposed model is evaluated on three larger test instances. Across nine combinatorial optimization problems, the generated models outperform the originals on 21 of 27 test instances, and on some problems solve more than two orders of magnitude faster. A comparison against non-agentic baselines that reuse the same validation and selection tools indicates that the gains stem from the agent's iterative diagnosis and repair, not merely from sampling several candidates. These results demonstrate that autonomous agentic methods can support the improvement of constraint models.

Florentina Voboril, Stefan Szeider

Algorithms and Complexity Group TU Wien, Vienna, Austria

Constraint Satisfaction Structural Constraints Large Language Model Agents Agentic Framework

Ask PDF

Apr 28, 2026math.OC

From Soliloquy to Agora: Memory-Enhanced LLM Agents with Decentralized Debate for Optimization Modeling

Optimization modeling underpins real-world decision-making in logistics, manufacturing, energy, and public services, but reliably solving such problems from natural-language requirements remains challenging for current large language models (LLMs). In this paper, we propose \emph{Agora-Opt}, a modular agentic framework for optimization modeling that combines decentralized debate with a read-write memory bank. Agora-Opt allows multiple agent teams to independently produce end-to-end solutions and reconcile them through an outcome-grounded debate protocol, while memory stores solver-verified artifacts and past disagreement resolutions to support training-free improvement over time. This design is flexible across both backbones and methods: it reduces base-model lock-in, transfers across different LLM families, and can be layered onto existing pipelines with minimal coupling. Across public benchmarks, Agora-Opt achieves the strongest overall performance among all compared methods, outperforming strong zero-shot LLMs, training-centric approaches, and prior agentic baselines. Further analyses show robust gains across backbone choices and component variants, and demonstrate that decentralized debate offers a structural advantage over centralized selection by enabling agents to refine candidate solutions through interaction and even recover correct formulations when all initial candidates are flawed. These results suggest that reliable optimization modeling benefits from combining collaborative cross-checking with reusable experience, and position Agora-Opt as a practical and extensible foundation for trustworthy optimization modeling assistance. Our code and data are available at https://github.com/CHIANGEL/Agora-Opt.

Jianghao Lin, Zi Ling, Chenyu Zhou +4

Antai College of Economics and Management, Shanghai Jiao Tong University · University of Chicago Booth School of Business · School of Economics and Management, Tsinghua University +1

Optimization Modeling Large Language Model Decisions Agentic Framework Decentralized Autonomous Organization

Ask PDF

Jul 12, 2026cs.AI

Opti-Agent-Bench: Benchmarking End-to-End Optimization R&D Agents on Real-World Business Problems

LLM-based agents are increasingly deployed to solve optimization problems, yet existing benchmarks evaluate them on pre-structured mathematical formulations that bypass the most critical challenge: translating complex business requirements into correct models and solve efficiently. We introduce Opti-Agent-Bench, an end-to-end benchmark that evaluates Large Language Models (LLMs) across the complete optimization R&D pipeline, from understanding business-language descriptions through mathematical modeling, algorithm selection, and code implementation, to solution report generation. Our design rests on three pillars: (1) businesssemantic authenticity with anti-template traps that defeat pattern matching; (2) modular evaluation with cross-module consistency checking across Problem Understanding, Formal Modeling, Implementation, and Reporting; and (3) the ORAC bi-level validity framework that simultaneously ensures task quality and scoring integrity. Across several industrialscale tasks spanning integer programming, robust optimization, stochastic programming, and non-convex optimization, we expose critical failure modes of current models, including constraint omission, model-code inconsistency, and report-implementation divergence, that remain invisible under conventional single-metric evaluation.

Yongchang Fu, Xinjie Huang, Chengjun Dai +3

Ding Talk, Alibaba Group, Hangzhou, China · Zhejiang University, Hangzhou, China

Optimization Modeling Agentic Benchmarks Large Language Model Benchmarks Report

Ask PDF