Choosing an appropriate representation of the environment for the underlying
decision-making process of the \gls{RL} agent is not always straightforward.
The state representation should be inclusive enough to allow the agent to
informatively decide on its actions and compact enough to increase sample
efficiency for policy training. Given this outlook, this work examines the
effect of various state representations in incentivizing the agent to solve a
specific robotic task: antipodal and planar object grasping. A continuum of
state representation abstractions is defined, starting from a model-based
approach with complete system knowledge, through hand-crafted numerical, to
image-based representations with decreasing level of induced task-specific
knowledge. We examine the effects of each representation in the ability of the
agent to solve the task in simulation and the transferability of the learned
policy to the real robot. The results show that RL agents using numerical
states can perform on par with non-learning baselines. Furthermore, we find
that agents using image-based representations from pre-trained environment
embedding vectors perform better than end-to-end trained agents, and
hypothesize that task-specific knowledge is necessary for achieving convergence
and high success rates in robot control. Supplementary material can be found at
the project webpage: this https URL

选择适当的环境表示对于强化学习代理的决策过程并不总是直观的，本研究考察了不同状态表示对于激励代理解决机器人任务的影响，结果显示使用数字状态的强化学习代理与非学习基线表现相当，而使用经过预训练环境嵌入向量的图像表示的代理表现更好，推测任务特定的知识对于实现收敛和高成功率的机器人控制是必要的。

表示抽象作为强化学习智能体的激励机制：一个机器人抓取的案例研究

Representation Abstractions as Incentives for Reinforcement Learning  Agents: A Robotic Grasping Case Study

Although an object may appear in numerous contexts, we often describe it in a
limited number of ways. Language allows us to abstract away visual variation to
represent and communicate concepts. Building on this intuition, we propose an
alternative approach to visual representation learning: using language
similarity to sample semantically similar image pairs for contrastive learning.
Our approach diverges from image-based contrastive learning by sampling view
pairs using language similarity instead of hand-crafted augmentations or
learned clusters. Our approach also differs from image-text contrastive
learning by relying on pre-trained language models to guide the learning rather
than directly minimizing a cross-modal loss. Through a series of experiments,
we show that language-guided learning yields better features than image-based
and image-text representation learning approaches.

本文提出一种新颖的视觉表达学习方法，使用语言相似性来对比学习语义相似的图像对，通过对比语言的相似性来采样图像视图对，避免手工增强和学习聚类。我们的方法通过预训练语言模型来引导学习，显示出比基于图像和图像 - 文本表征学习方法更好的特征。