In this paper, we explore how we can build upon the data and models of
Internet images and use them to adapt to robot vision without requiring any
extra labels. We present a framework called Self-supervised Embodied Active
Learning (SEAL). It utilizes perception models trained on internet images to
learn an active exploration policy. The observations gathered by this
exploration policy are labelled using 3D consistency and used to improve the
perception model. We build and utilize 3D semantic maps to learn both action
and perception in a completely self-supervised manner. The semantic map is used
to compute an intrinsic motivation reward for training the exploration policy
and for labelling the agent observations using spatio-temporal 3D consistency
and label propagation. We demonstrate that the SEAL framework can be used to
close the action-perception loop: it improves object detection and instance
segmentation performance of a pretrained perception model by just moving around
in training environments and the improved perception model can be used to
improve Object Goal Navigation.

该论文介绍了一个名为自助式躯体活动学习（Self-supervised Embodied Active Learning，SEAL）的框架，它使用训练于互联网图像的感知模型学习主动探索策略，并使用 3D 一致性对收集的观察结果进行标记，并利用 3D 语义地图在完全自我监督的情况下学习行动和感知，从而可以提高对目标的检测和分割的性能。