The 3D Bin Packing Problem (3D-BPP) is one of the most demanded yet
challenging problems in industry, where an agent must pack variable size items
delivered in sequence into a finite bin with the aim to maximize the space
utilization. It represents a strongly NP-Hard optimization problem such that no
solution has been offered to date with high performance in space utilization.
In this paper, we present a new reinforcement learning (RL) framework for a
3D-BPP solution for improving performance. First, a buffer is introduced to
allow multi-item action selection. By increasing the degree of freedom in
action selection, a more complex policy that results in better packing
performance can be derived. Second, we propose an agnostic data augmentation
strategy that exploits both bin item symmetries for improving sample
efficiency. Third, we implement a model-based RL method adapted from the
popular algorithm AlphaGo, which has shown superhuman performance in zero-sum
games. Our adaptation is capable of working in single-player and score based
environments. In spite of the fact that AlphaGo versions are known to be
computationally heavy, we manage to train the proposed framework with a single
thread and GPU, while obtaining a solution that outperforms the
state-of-the-art results in space utilization.

本文提出了一种基于强化学习框架的三维装箱问题（3D-BPP）解决方案，其中引入了一个缓冲区以允许多物品动作选择，并提出了一种采用容器物品对称性的数据增强策略，采用类似于 AlphaGo 的模型 RL 方法进行实现，用单线程和 GPU 训练该框架，得出了在空间利用率方面优于现有成果的解决方案。

在线 3D 装箱强化学习解决方案（附缓冲区）

Online 3D Bin Packing Reinforcement Learning Solution with Buffer

In this paper, a new type of 3D bin packing problem (BPP) is proposed, in
which a number of cuboid-shaped items must be put into a bin one by one
orthogonally. The objective is to find a way to place these items that can
minimize the surface area of the bin. This problem is based on the fact that
there is no fixed-sized bin in many real business scenarios and the cost of a
bin is proportional to its surface area. Our research shows that this problem
is NP-hard. Based on previous research on 3D BPP, the surface area is
determined by the sequence, spatial locations and orientations of items. Among
these factors, the sequence of items plays a key role in minimizing the surface
area. Inspired by recent achievements of deep reinforcement learning (DRL)
techniques, especially Pointer Network, on combinatorial optimization problems
such as TSP, a DRL-based method is applied to optimize the sequence of items to
be packed into the bin. Numerical results show that the method proposed in this
paper achieve about 5% improvement than heuristic method.

本文提出一种新型三维装箱问题，利用深度强化学习算法中的 Pointer Network 方法来优化装箱物品的序列，以期在不固定大小容器的场景下，通过最小化表面积来减少容器成本。数值结果显示该方法较启发式方法改进了约 5%。