Deep reinforcement learning (DRL) has shown remarkable success in simulation
domains, yet its application in designing robot controllers remains limited,
due to its single-task orientation and insufficient adaptability to
environmental changes. To overcome these limitations, we present a novel
adaptive agent that leverages transfer learning techniques to dynamically adapt
policy in response to different tasks and environmental conditions. The
approach is validated through the blimp control challenge, where multitasking
capabilities and environmental adaptability are essential. The agent is trained
using a custom, highly parallelized simulator built on IsaacGym. We perform
zero-shot transfer to fly the blimp in the real world to solve various tasks.
We share our code at
https://github.com/robot-perception-group/adaptive\_agent/.

深度强化学习在模拟环境中取得了显著的成功，但在设计机器人控制器方面的应用仍然有限，由于其单任务导向性和对环境变化的适应能力不足。为了克服这些限制，我们提出了一种新颖的自适应智能体，利用迁移学习技术在不同任务和环境条件下动态调整策略。该方法经过了齐腾博仿真挑战的验证，其中多任务能力和环境适应能力至关重要。智能体使用基于 IsaacGym 的定制化高度并行化的模拟器进行训练，通过零样本迁移在真实世界中飞行气球以解决各种任务。我们在 https://github.com/robot-perception-group/adaptive_agent/ 上分享我们的代码。

机器人控制的自适应强化学习

Adaptive Reinforcement Learning for Robot Control

Developing robot controllers capable of achieving dexterous nonprehensile
manipulation, such as pushing an object on a table, is challenging. The
underactuated and hybrid-dynamics nature of the problem, further complicated by
the uncertainty resulting from the frictional interactions, requires
sophisticated control behaviors. Reinforcement Learning (RL) is a powerful
framework for developing such robot controllers. However, previous RL
literature addressing the nonprehensile pushing task achieves low accuracy,
non-smooth trajectories, and only simple motions, i.e. without rotation of the
manipulated object. We conjecture that previously used unimodal exploration
strategies fail to capture the inherent hybrid-dynamics of the task, arising
from the different possible contact interaction modes between the robot and the
object, such as sticking, sliding, and separation. In this work, we propose a
multimodal exploration approach through categorical distributions, which
enables us to train planar pushing RL policies for arbitrary starting and
target object poses, i.e. positions and orientations, and with improved
accuracy. We show that the learned policies are robust to external disturbances
and observation noise, and scale to tasks with multiple pushers. Furthermore,
we validate the transferability of the learned policies, trained entirely in
simulation, to a physical robot hardware using the KUKA iiwa robot arm. See our
supplemental video: this https URL

通过多模态探索方法，提出了用于任意起始和目标物体姿态的平面推动强化学习策略，该策略在无模拟故障和观测噪声的情况下具备可靠性，并能应用于具有多个推动装置的任务，同时验证了该策略在物理机器人上的可转移性。

通过多模态分类探索进行强化学习的非抓取平面操作

Nonprehensile Planar Manipulation through Reinforcement Learning with  Multimodal Categorical Exploration

We introduce an approximate model for predicting the net contact wrench
between nominally rigid objects for use in simulation, control, and state
estimation. The model combines and generalizes two ideas: a bed of springs (an
"elastic foundation") and hydrostatic pressure. In this model, continuous
pressure fields are computed offline for the interior of each nominally rigid
object. Unlike hydrostatics or elastic foundations, the pressure fields need
not satisfy mechanical equilibrium conditions. When two objects nominally
overlap, a contact surface is defined where the two pressure fields are equal.
This static pressure is supplemented with a dissipative rate-dependent pressure
and friction to determine tractions on the contact surface. The contact wrench
between pairs of objects is an integral of traction contributions over this
surface. The model evaluates much faster than elasticity-theory models, while
showing the essential trends of force, moment, and stiffness increase with
contact load. It yields continuous wrenches even for non-convex objects and
coarse meshes. The method shows promise as sufficiently fast, accurate, and
robust for design-in-simulation of robot controllers.

本文提出了一种近似模型，用于预测模拟、控制和状态估计中的名义硬物体之间的净接触力，其结合并推广了弹簧床 (一种 “弹性基础”) 和静压力两种思想，该模型可以更快速地评估，具有足够快、准确和鲁棒的特点，适用于机器人控制器的设计和模拟。