Diffusion Reinforcement Learning Based Online 3D Bin Packing Spatial Strategy Optimization

Jie Han; Tong Li; Qingyang Xu; Yong Song; Bao Pang; Xianfeng Yuan

cs.ROApr 13, 2026

Diffusion Reinforcement Learning Based Online 3D Bin Packing Spatial Strategy Optimization

Authors: Jie Han, Tong Li, Qingyang Xu, Yong Song, Bao Pang, Xianfeng Yuan

Organizations: School of Mechanical, Electrical, and Information Engineering, Shandong University, Weihai 264209, China · Shandong Key Laboratory of Intelligent Electronic Packaging Testing and Application, Shandong University, Weihai 264209, China

Abstract

The online 3D bin packing problem is important in logistics, warehousing and intelligent manufacturing, with solutions shifting to deep reinforcement learning (DRL) which faces challenges like low sample efficiency. This paper proposes a diffusion reinforcement learning-based algorithm, using a Markov decision chain for packing modeling, height map-based state representation and a diffusion model-based actor network. Experiments show it significantly improves the average number of packed items compared to state-of-the-art DRL methods, with excellent application potential in complex online scenarios.

RL for Combinatorial Optimization

Reinforcement learning for combinatorial optimization learns sequential decisions and search strategies for difficult discrete problems.

See all 106 papers

Spatial Heuristic Strategy based Diffusion Reinforcement Learning for Online 3D Bin Packing
Oct 23, 2026Tong Li, Qingyang XuReinforcement Learning RL for Combinatorial Optimization
School of Airspace Science and Engineering, Shandong University, Weihai 264209, China, and also with the Shandong Key Laboratory of Intelligent Electronic Packaging Testing and Application, Shandong University, Weihai 264209.
Operationally Guided Placement-Aware Learning for Industrial Online 3D Bin Packing
Jul 30, 2026Dheeraj Poolavaram, Aanchal Rajesh Chugh, Sebastian DornRL for Combinatorial Optimization
TTZ Landsberg am Lech, Technische Hochschule Augsburg, Augsburg, Germany
Deep Reinforcement Learning on Item-Compatibility Graphs for One-Dimensional Bin Packing
Sep 21, 2026M. Aslı AydınCombinatorial Optimization RL for Combinatorial Optimization
Department of Management Engineering, Bahçeşehir University, Istanbul, Türkiye

Oct 23, 2026J/K move · Enter open · S save

Spatial Heuristic Strategy based Diffusion Reinforcement Learning for Online 3D Bin Packing

Tong Li, Qingyang Xu

School of Airspace Science and Engineering, Shandong University, Weihai 264209, China, and also with the Shandong Key Laboratory of Intelligent Electronic Packaging Testing and Application, Shandong University, Weihai 264209.

Reinforcement Learning RL for Combinatorial Optimization

Open paper AskPDF arXiv

Traditional heuristic methods have accumulated mature experience in core objectives, such as space utilization and layout compactness of 3D bin packing, relying on artificially designed rule-based logic. However, they exhibit obvious limitations in generalization ability and adaptability to dynamic scenarios. Diffusion reinforcement learning, which integrates the spatial search capability of reinforcement learning and the data distribution modeling capability of diffusion learning, has achieved certain progress in action distribution modeling and decision compliance constraints. Nevertheless, there remain issues including insufficient perception of the spatial topological structure in 3D bin packing and limited layout optimization capability. To address these problems, we propose a diffusion reinforcement learning method integrated with spatial heuristic strategies, which combines classical bin packing heuristic rules with a diffusion reinforcement learning framework. Specifically, it compresses the action space through heuristic action pre-screening, introduces a graph neural network to model the spatial structure, and designs a composite reward function incorporating flatness to suppress spatial fragmentation, thereby realizing the organic integration of prior knowledge and adaptive optimization. Experimental results demonstrate that the pro

Abstract

RL for Combinatorial Optimization

Explore similar work

Spatial Heuristic Strategy based Diffusion Reinforcement Learning for Online 3D Bin Packing

Operationally Guided Placement-Aware Learning for Industrial Online 3D Bin Packing

Deep Reinforcement Learning on Item-Compatibility Graphs for One-Dimensional Bin Packing