AI-Assisted Discovery of Convex Relaxations via Dual Agents

The runtime of Constraint Programming (CP) solvers is highly sensitive to modeling choices, such as symmetry breaking, implied constraints, global constraints, constraint reformulation, and variable representation. Improving these constraint models has traditionally required human expertise, and existing automated reformulation systems are restricted to a predefined library of hand-crafted transformation rules. We introduce an agentic framework that instead reformulates a constraint model from an open-ended space and establishes correctness empirically rather than by construction: a Large Language Model (LLM) agent, given a model and three training instances, proposes alternative formulations, validates each by injecting its solution back into the original model, and diagnoses and repairs failures, returning the best variant it finds in a median of about fifteen minutes. The models are expressed in the CPMpy modeling library, and each proposed model is evaluated on three larger test instances. Across nine combinatorial optimization problems, the generated models outperform the originals on 21 of 27 test instances, and on some problems solve more than two orders of magnitude faster. A comparison against non-agentic baselines that reuse the same validation and selection tools indicates that the gains stem from the agent's iterative diagnosis and repair, not merely from sampling several candidates. These results demonstrate that autonomous agentic methods can support the improvement of constraint models.

Many optimization problems arise repeatedly from a fixed but unknown distribution. Even when the worst-case problem is hard, this distribution may carry reusable structure, such as recurring geometry, decompositions, or resource patterns. We study how to infer such structure from sample instances and compile it into solver code that runs faster on future instances while preserving solution quality. Our central abstraction is a \emph{solver hint}: distribution-specific structure inferred from samples and used to specialize a solver. We prove that the empirically fastest sample-consistent solver generalizes in both correctness and runtime over fixed solver libraries, and that identifiable hints can be recovered from polynomially many samples. We instantiate the framework with LLM code agents on $21$ combinatorial-optimization distributions across $7$ problem classes. The synthesized solvers reach mean normalized quality $0.971$ while running orders of magnitude faster than classical heuristics, Gurobi, and time-limited exact backends, though they do not dominate every baseline on every family. Against LLM synthesis baselines, they are faster than one-shot Codex, one-shot Claude Code, and a best-of-$5$ open-model variant; they improve quality over Claude Code and best-of-$5$, while nearly matching Codex quality and running substantially faster. This isolates the contribution of the iterative synthesis loop without claiming uniform domination over every LLM baseline. On the PACE 2025 Dominating Set private instances, the synthesized solver is valid on all $100$ graphs and runs roughly $75\times$--$125\times$ faster than released competition solvers, within a few percent of their solution size. These results suggest LLM agents can discover distribution-specific computational shortcuts and compile them into efficient solver code.

Many neural network (NN) verification systems represent the network's input-output relation as a constraint program. Sound and complete, representations involve integer constraints, for simulating the activations. Recent works convexly relax the integer constraints, improving performance, at the cost of soundness. Convex relaxations consider outputs that are unreachable by the original network. We study the worst case divergence between the original network and its convex relaxations; both qualitatively and quantitatively. The relaxations' space forms a lattice, where the top element corresponds to a full relaxation, with every neuron linearized. The bottom element corresponds to the original network. We provide analytical upper and lower bounds for the $\ell_\infty$-distance between the fully relaxed and original outputs. This distance grows exponentially, w.r.t. the network's depth, and linearly w.r.t. the input's radius. The misclassification probability exhibits a step-like behavior, w.r.t. input radius. Our results are supported by experiments on MNIST, Fashion MNIST and random networks.