We present a novel learning framework to solve the transport-and-packing
(TAP) problem in 3D. It constitutes a full solution pipeline from partial
observations of input objects via RGBD sensing and recognition to final box
placement, via robotic motion planning, to arrive at a compact packing in a
target container. The technical core of our method is a neural network for TAP,
trained via reinforcement learning (RL), to solve the NP-hard combinatorial
optimization problem. Our network simultaneously selects an object to pack and
determines the final packing location, based on a judicious encoding of the
continuously evolving states of partially observed source objects and available
spaces in the target container, using separate encoders both enabled with
attention mechanisms. The encoded feature vectors are employed to compute the
matching scores and feasibility masks of different pairings of box selection
and available space configuration for packing strategy optimization. Extensive
experiments, including ablation studies and physical packing execution by a
real robot (Universal Robot UR5e), are conducted to evaluate our method in
terms of its design choices, scalability, generalizability, and comparisons to
baselines, including the most recent RL-based TAP solution. We also contribute
the first benchmark for TAP which covers a variety of input settings and
difficulty levels.

我们提出了一个新的学习框架来解决 3D 中的运输和装箱问题，其中的技术核心是使用强化学习训练的神经网络，通过 RGBD 感知和识别，机器人运动规划，来实现在目标容器中进行紧密的装箱。