Choosing an appropriate representation of the environment for the underlying
decision-making process of the \gls{RL} agent is not always straightforward.
The state representation should be inclusive enough to allow the agent to
informatively decide on its actions and compact enough to increase sample
efficiency for policy training. Given this outlook, this work examines the
effect of various state representations in incentivizing the agent to solve a
specific robotic task: antipodal and planar object grasping. A continuum of
state representation abstractions is defined, starting from a model-based
approach with complete system knowledge, through hand-crafted numerical, to
image-based representations with decreasing level of induced task-specific
knowledge. We examine the effects of each representation in the ability of the
agent to solve the task in simulation and the transferability of the learned
policy to the real robot. The results show that RL agents using numerical
states can perform on par with non-learning baselines. Furthermore, we find
that agents using image-based representations from pre-trained environment
embedding vectors perform better than end-to-end trained agents, and
hypothesize that task-specific knowledge is necessary for achieving convergence
and high success rates in robot control. Supplementary material can be found at
the project webpage: this https URL

选择适当的环境表示对于强化学习代理的决策过程并不总是直观的，本研究考察了不同状态表示对于激励代理解决机器人任务的影响，结果显示使用数字状态的强化学习代理与非学习基线表现相当，而使用经过预训练环境嵌入向量的图像表示的代理表现更好，推测任务特定的知识对于实现收敛和高成功率的机器人控制是必要的。

表示抽象作为强化学习智能体的激励机制：一个机器人抓取的案例研究

Representation Abstractions as Incentives for Reinforcement Learning  Agents: A Robotic Grasping Case Study

Uncertainty in perception, actuation, and the environment often require
multiple attempts for a robotic task to be successful. We study a class of
problems providing (1) low-entropy indicators of terminal success / failure,
and (2) unreliable (high-entropy) data to predict the final outcome of an
ongoing task. Examples include a robot trying to connect with a charging
station, parallel parking, or assembling a tightly-fitting part. The ability to
restart after predicting failure early, versus simply running to failure, can
significantly decrease the makespan, that is, the total time to completion,
with the drawback of potentially short-cutting an otherwise successful
operation. Assuming task running times to be Poisson distributed, and using a
Markov Jump process to capture the dynamics of the underlying Markov Decision
Process, we derive a closed form solution that predicts makespan based on the
confusion matrix of the failure predictor. This allows the robot to learn
failure prediction in a production environment, and only adopt a preemptive
policy when it actually saves time. We demonstrate this approach using a
robotic peg-in-hole assembly problem using a real robotic system. Failures are
predicted by a dilated convolutional network based on force-torque data,
showing an average makespan reduction from 101s to 81s (N=120, p<0.05). We
posit that the proposed algorithm generalizes to any robotic behavior with an
unambiguous terminal reward, with wide ranging applications on how robots can
learn and improve their behaviors in the wild.

通过机器学习中的失效预测和可重新启动的策略来降低机器人完成任务的时间和提高其行为表现。

机器人装配及其他试错任务的最优决策

Optimal decision making in robotic assembly and other trial-and-error tasks

Generating dexterous grasping has been a long-standing and challenging
robotic task. Despite recent progress, existing methods primarily suffer from
two issues. First, most prior arts focus on a specific type of robot hand,
lacking the generalizable capability of handling unseen ones. Second, prior
arts oftentimes fail to rapidly generate diverse grasps with a high success
rate. To jointly tackle these challenges with a unified solution, we propose
GenDexGrasp, a novel hand-agnostic grasping algorithm for generalizable
grasping. GenDexGrasp is trained on our proposed large-scale multi-hand
grasping dataset MultiDex synthesized with force closure optimization. By
leveraging the contact map as a hand-agnostic intermediate representation,
GenDexGrasp efficiently generates diverse and plausible grasping poses with a
high success rate and can transfer among diverse multi-fingered robotic hands.
Compared with previous methods, GenDexGrasp achieves a three-way trade-off
among success rate, inference speed, and diversity. Code is available at
this https URL

提出了一种基于多型机械手的手无偏见抓取算法 GenDexGrasp，该算法利用接触映射作为机械手无偏见中间表示，采用力闭合优化综合生成了大规模的多手抓取数据集 MultiDex。与先前的方法相比，GenDexGrasp 在成功率、推断速度和多样性之间实现了三重平衡。