Generic re-usable pre-trained image representation encoders have become a
standard component of methods for many computer vision tasks. As visual
representations for robots however, their utility has been limited, leading to
a recent wave of efforts to pre-train robotics-specific image encoders that are
better suited to robotic tasks than their generic counterparts. We propose
Scene Objects From Transformers, abbreviated as SOFT, a wrapper around
pre-trained vision transformer (PVT) models that bridges this gap without any
further training. Rather than construct representations out of only the final
layer activations, SOFT individuates and locates object-like entities from PVT
attentions, and describes them with PVT activations, producing an
object-centric embedding. Across standard choices of generic pre-trained vision
transformers PVT, we demonstrate in each case that policies trained on
SOFT(PVT) far outstrip standard PVT representations for manipulation tasks in
simulated and real settings, approaching the state-of-the-art robotics-aware
representations. Code, appendix and videos:
this https URL

通过使用预训练图像表示编码器和视觉转换器，Scene Objects From Transformers (SOFT) 构建了一个对象中心化嵌入，无需进一步训练即可优化机器人的操作任务。