Training end-to-end deep robot policies requires a lot of domain-, task-, and
hardware-specific data, which is often costly to provide. In this work, we
propose to tackle this issue by employing a deep neural network with a modular
architecture, consisting of separate perception, policy, and trajectory parts.
Each part of the system is trained fully on synthetic data or in simulation.
The data is exchanged between parts of the system as low-dimensional latent
representations of affordances and trajectories. The performance is then
evaluated in a zero-shot transfer scenario using Franka Panda robot arm.
Results demonstrate that a low-dimensional representation of scene affordances
extracted from an RGB image is sufficient to successfully train manipulator
policies. We also introduce a method for affordance dataset generation, which
is easily generalizable to new tasks, objects and environments, and requires no
manual pixel labeling.

采用模块化神经网络体系结构，利用合成数据和模拟技术，通过低维潜在表示生成场景可用性，成功训练机械臂政策。同时，提出了一种数据集生成方法，可轻松推广到新任务、对象和环境，无需手动像素标记。