stat.ML · 2605.02461 Copy arXiv ID · May 4, 2026 Save Middle-mile logistics through the lens of goal-conditioned reinforcement learning Authors: Onno Eberhard , Thibaut Cuvelier , Michal Valko , Bruno De Backer
Organizations: Max Planck Institute for Intelligent Systems, Tübingen, Germany · University of Tübingen · Google Research · Google DeepMind
Abstract Middle-mile logistics describes the problem of routing parcels through a network of hubs linked by trucks with finite capacity. We rephrase this as a multi-object goal-conditioned MDP. Our method combines graph neural networks with model-free RL, extracting small feature graphs from the environment state.
Explore similar work
May 27, 2026 · cs.LG J/K move · Enter open · S save
Kaiqiang Ke, Shenghong He, Chengdong Xu, Yuheng Luo +2
1Sun Yat-sen University · 2Pengcheng Laboratory
Offline goal-conditioned reinforcement learning (GCRL) is challenging in long-horizon tasks, where distant state--goal pairs provide weak supervision and value estimates become vulnerable to accumulated bootstrapping errors. Hierarchical methods mitigate this difficulty by introducing intermediate subgoals, but fixed temporal abstractions or fixed hierarchy depths can be mismatched to state--goal pairs with different reachability horizons. We propose Coarse-to-Fine Hierarchical Goal Reinforcement Learning (CFHRL), a fully offline GCRL framework that adaptively refines distant goals before execution. Starting from the final goal, CFHRL recursively proposes intermediate targets, trained from replay-supported candidates, and stops refinement once the current target is estimated to be locally executable by a learned reachability cost. The key idea is that a subgoal need not be an exact midpoint or globally optimal waypoint; it only needs to provide reliable progress and reduce the remaining reaching difficulty, enabling subsequent refinement over shorter horizons. A stylized analysis further supports the robustness of approximate recursive contraction. Experiments on OGBench show substantial gains on several long-horizon tasks, with ablations validating the proposed refinement and stopping mechanisms