Construct, Merge, Solve & Adapt with Reinforcement Learning for the min-max Multiple Traveling Salesman Problem
Organizations: Artificial Intelligence Research Institute (IIIA-CSIC), Bellaterra, Catalunya, Spain · Universitat Politècnica de Catalunya (UPC), Barcelona, Catalunya, Spain
Abstract
The Multiple Traveling Salesman Problem (mTSP) extends the Traveling Salesman Problem to m tours that start and end at a common depot and jointly visit all customers exactly once. In the min-max variant, the objective is to minimize the longest tour, reflecting workload balance. We propose a hybrid approach, Construct, Merge, Solve & Adapt with Reinforcement Learning (RL-CMSA), for the symmetric single-depot min-max mTSP. The method iteratively constructs diverse solutions using probabilistic clustering guided by learned pairwise q-values, merges routes into a compact pool, solves a restricted set-covering MILP, and refines solutions via inter-route remove, shift, and swap moves. The q-values are updated by reinforcing city-pair co-occurrences in high-quality solutions, while the pool is adapted through ageing and pruning. This combination of exact optimization and reinforcement-guided construction balances exploration and exploitation. Computational results on random and TSPLIB instances show that RL-CMSA consistently finds (near-)best solutions and outperforms a state-of-the-art hybrid genetic algorithm under comparable time limits, especially as instance size and the number of salesmen increase.
Figures & tables
| Parameters | Considered Domain | Tuning results | |||
|---|---|---|---|---|---|
| n | ||||||||||||||||||||||||||
| HGA | RL-CMSA | HGA | RL-CMSA | HGA | RL-CMSA | |||||||||||||||||||||
| id | mean | #b | time | mean | #b | time | best | mean | #b | time | mean | #b | time | best | mean | #b | time | mean | #b | time | best | |||||
| 0 | 5.0575160 | 40 | 0.87 | 5.0575160 | 40 | 0.04 | 5.0575160 | 7.0156623 | 39 | 5.60 | 7.0156453 | 40 | 1.41 | 7.0156453 | 9.6706542 | 2 | 92.60 | 9.6904399 | 0 | 87.50 | 9.6328396 | |||||
| 1 | 5.3701311 | 40 | 1.52 | 5.3701311 | 40 | 0.09 | 5.3701311 | 6.8430869 | 40 | 6.88 | 6.8430869 | 40 | 0.34 | 6.8430869 | 9.3918357 | 11 | 90.76 | 9.3618625 | 4 | 77.57 | 9.3195580 | |||||
| 2 | 5.3316439 | 40 | 1.38 | 5.3316439 | 40 | 0.04 | 5.3316439 | 7.0299344 | 40 | 12.08 | 7.0299344 | 40 | 2.25 | 7.0299344 | 9.4244805 | 2 | 61.20 | 9.4208673 | 4 | 86.93 | 9.4005750 | |||||
| 3 | 5.2586716 | 40 | 1.27 | 5.2586716 | 40 | 0.04 | 5.2586716 | 6.9746008 | 27 | 26.94 | 6.9623471 | 40 | 1.91 | 6.9623471 | 9.4609795 | 1 | 84.39 | 9.4815937 | 0 | 89.62 | 9.4181420 | |||||
| n | ||||||
|---|---|---|---|---|---|---|
| m | -value | -value | -value | |||
| 0.0000 | 1.000e+00 | -1.0000 | 4.883e-04 *** | 0.3714 | 5.000e-01 | |
| -1.0000 | 1.000e+00 | -1.0000 | 4.196e-05 *** | -1.0000 | 2.289e-05 *** | |
| -1.0000 | 5.000e-01 | -1.0000 | 2.747e-04 *** | -0.8762 | 1.175e-03 ** | |
| -1.0000 | 3.906e-02 * | -0.8242 | 3.662e-02 * | -1.0000 | 4.196e-05 *** | |
| eil51 | berlin52 | eil76 | rat99 | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HGA | RL-CMSA | HGA | RL-CMSA | HGA | RL-CMSA | HGA | RL-CMSA | ||||||||||||||||||||
| m | mean | #b | mean | #b | best | mean | #b | mean | #b | best | mean | #b | mean | #b | best | mean | #b | mean | #b | best | |||||||
| 222.75794 | 37 | 222.73337 | 40 | 222.73337 | 4110.21310 | 40 | 4110.21310 | 40 | 4110.21310 | 281.22379 | 25 | 280.85394 | 40 | 280.85394 | 665.99090 | 40 | 665.99090 | 40 | 665.99090 | ||||||||
| 159.57151 | 40 | 159.57151 | 40 | 159.57151 | 3069.58569 | 40 | 3069.58569 | 40 | 3069.58569 | 159.50091 | 40 | 159.50091 | 40 | 159.50091 | 450.33361 | 0 | 450.29462 | 24 | 450.27258 | ||||||||
| 118.13375 | 40 | 118.13375 | 40 | 118.13375 | 2440.92196 | 40 | 2440.92196 | 40 | 2440.92196 | 127.56176 | 39 | 127.56175 | 40 | 127.56175 | 436.44014 | 40 | 436.44014 | 40 | 436.44014 | ||||||||
| 112.07141 | 40 | 112.07141 | 40 | 112.07141 | 2440.92196 | 40 | 2440.92196 | 40 | 2440.92196 | 127.56175 | 40 | 127.56175 | 40 | 127.56175 | 436.44014 | 40 | 436.44014 | 40 | 436.44014 | ||||||||
Traveling Salesperson Problem
The Traveling Salesperson Problem studies algorithms for finding shortest tours that visit every city exactly once.
See all 38 papers