We investigate policy transfer using image-to-semantics translation to
mitigate learning difficulties in vision-based robotics control agents. This
problem assumes two environments: a simulator environment with semantics, that
is, low-dimensional and essential information, as the state space, and a
real-world environment with images as the state space. By learning mapping from
images to semantics, we can transfer a policy, pre-trained in the simulator, to
the real world, thereby eliminating real-world on-policy agent interactions to
learn, which are costly and risky. In addition, using image-to-semantics
mapping is advantageous in terms of the computational efficiency to train the
policy and the interpretability of the obtained policy over other types of
sim-to-real transfer strategies. To tackle the main difficulty in learning
image-to-semantics mapping, namely the human annotation cost for producing a
training dataset, we propose two techniques: pair augmentation with the
transition function in the simulator environment and active learning. We
observed a reduction in the annotation cost without a decline in the
performance of the transfer, and the proposed approach outperformed the
existing approach without annotation.

本研究探讨采用图像到语义翻译技术进行政策传递，缓解基于视觉的机器人控制代理的学习困难问题。通过学习从图像到语义的映射，我们可以将在模拟器中预先训练的政策传递到现实世界中，从而消除学习花费和风险高的现实世界上策略的实时交互学习。此外，使用图像到语义映射在训练策略时具有计算效率和可解释性优势。我们提出了两种技术：在模拟器环境中使用转换函数的配对增强技术和主动学习，以解决图像到语义映射中的主要困难，即为产生训练数据的人工注释成本。我们观察到注释成本的降低，而传递的性能不会下降。因此，所提出的方法优于现有的未经注释的方法。