The ability to autonomously learn behaviors via direct interactions in
uninstrumented environments can lead to generalist robots capable of enhancing
productivity or providing care in unstructured settings like homes. Such
uninstrumented settings warrant operations only using the robot's
proprioceptive sensor such as onboard cameras, joint encoders, etc which can be
challenging for policy learning owing to the high dimensionality and partial
observability issues. We propose RRL: Resnet as representation for
Reinforcement Learning -- a straightforward yet effective approach that can
learn complex behaviors directly from proprioceptive inputs. RRL fuses features
extracted from pre-trained Resnet into the standard reinforcement learning
pipeline and delivers results comparable to learning directly from the state.
In a simulated dexterous manipulation benchmark, where the state of the art
methods fail to make significant progress, RRL delivers contact rich behaviors.
The appeal of RRL lies in its simplicity in bringing together progress from the
fields of Representation Learning, Imitation Learning, and Reinforcement
Learning. Its effectiveness in learning behaviors directly from visual inputs
with performance and sample efficiency matching learning directly from the
state, even in complex high dimensional domains, is far from obvious.

我们提出了 RRL（使用 Resnet 作为 Reinforcement Learning 的表示）作为一种简单而有效的方法，可以直接从固有输入中学习复杂的行为，其在高维领域中的学习性能和样本效率甚至可以与直接从状态中学习相匹配，尤其在模拟的机器人灵巧操作基准中表现出接触丰富的行为。

RRL：ResNet 用于强化学习的表示

RRL: Resnet as representation for Reinforcement Learning

Mobile robots operating in unknown urban environments encounter a wide range
of complex terrains to which they must adapt their planned trajectory for safe
and efficient navigation. Most existing approaches utilize supervised learning
to classify terrains from either an exteroceptive or a proprioceptive sensor
modality. However, this requires a tremendous amount of manual labeling effort
for each newly encountered terrain as well as for variations of terrains caused
by changing environmental conditions. In this work, we propose a novel terrain
classification framework leveraging an unsupervised proprioceptive classifier
that learns from vehicle-terrain interaction sounds to self-supervise an
exteroceptive classifier for pixel-wise semantic segmentation of images. To
this end, we first learn a discriminative embedding space for vehicle-terrain
interaction sounds from triplets of audio clips formed using visual features of
the corresponding terrain patches and cluster the resulting embeddings. We
subsequently use these clusters to label the visual terrain patches by
projecting the traversed tracks of the robot into the camera images. Finally,
we use the sparsely labeled images to train our semantic segmentation network
in a weakly supervised manner. We present extensive quantitative and
qualitative results that demonstrate that our proprioceptive terrain classifier
exceeds the state-of-the-art among unsupervised methods and our self-supervised
exteroceptive semantic segmentation model achieves a comparable performance to
supervised learning with manually labeled data.

提出了一种利用无监督感知分类器自我监督外感图像的像素级语义分割的新型地形分类框架，通过学习极少标注图像的弱监督训练，能够在未知城市环境中为移动机器人提供更安全和高效的导航。