cs.LGOct 2, 2025

Black-Box Combinatorial Optimization with Order-Invariant Reinforcement Learning

Authors: Olivier Goudet, Quentin Suire, Adrien Goëffon, Frédéric Saubion, Sylvain Lamprier

Organizations: LERIA, Université d’Angers, 2 Boulevard Lavoisier Angers 49045 France.

Abstract

We introduce an order-invariant reinforcement learning framework for black-box combinatorial optimization. Classical estimation-of-distribution algorithms (EDAs) often rely on learning explicit variable dependency graphs, which can be costly and fail to capture complex interactions efficiently. In contrast, we parameterize a multivariate autoregressive generative model trained without a fixed variable ordering. By sampling random generation orders during training, a form of information-preserving dropout, the model is encouraged to be invariant to variable order, promoting search-space diversity, and shaping the model to focus on the most relevant variable dependencies, improving sample efficiency. We adapt Group Relative Policy Optimization (GRPO) to this setting, providing stable policy-gradient updates from scale-invariant advantages. Across a wide range of benchmark algorithms and problem instances of varying sizes, our method frequently achieves the best performance and consistently avoids catastrophic failures.

Figures & tables

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Stein Variational Black-Box Combinatorial Optimization

    Apr 17, 2026Thomas Landais, Olivier Goudet, Adrien Goëffon +2Stochastic OptimizationBlack-Box Optimization

  2. Beyond Inference-Time Search: Reinforcement Learning Synthesizes Reusable Solvers

    May 18, 2026Soheyl Massoudi, Gabriel Apaza, Milad Habibi +1RL for Code GenerationLLM-Based Program Synthesis