Deep reinforcement learning (DRL) has shown remarkable success in simulation
domains, yet its application in designing robot controllers remains limited,
due to its single-task orientation and insufficient adaptability to
environmental changes. To overcome these limitations, we present a novel
adaptive agent that leverages transfer learning techniques to dynamically adapt
policy in response to different tasks and environmental conditions. The
approach is validated through the blimp control challenge, where multitasking
capabilities and environmental adaptability are essential. The agent is trained
using a custom, highly parallelized simulator built on IsaacGym. We perform
zero-shot transfer to fly the blimp in the real world to solve various tasks.
We share our code at
https://github.com/robot-perception-group/adaptive\_agent/.

深度强化学习在模拟环境中取得了显著的成功，但在设计机器人控制器方面的应用仍然有限，由于其单任务导向性和对环境变化的适应能力不足。为了克服这些限制，我们提出了一种新颖的自适应智能体，利用迁移学习技术在不同任务和环境条件下动态调整策略。该方法经过了齐腾博仿真挑战的验证，其中多任务能力和环境适应能力至关重要。智能体使用基于 IsaacGym 的定制化高度并行化的模拟器进行训练，通过零样本迁移在真实世界中飞行气球以解决各种任务。我们在 https://github.com/robot-perception-group/adaptive_agent/ 上分享我们的代码。

机器人控制的自适应强化学习

Adaptive Reinforcement Learning for Robot Control

Foundation models have shown impressive adaptation and scalability in
supervised and self-supervised learning problems, but so far these successes
have not fully translated to reinforcement learning (RL). In this work, we
demonstrate that training an RL agent at scale leads to a general in-context
learning algorithm that can adapt to open-ended novel embodied 3D problems as
quickly as humans. In a vast space of held-out environment dynamics, our
adaptive agent (AdA) displays on-the-fly hypothesis-driven exploration,
efficient exploitation of acquired knowledge, and can successfully be prompted
with first-person demonstrations. Adaptation emerges from three ingredients:
(1) meta-reinforcement learning across a vast, smooth and diverse task
distribution, (2) a policy parameterised as a large-scale attention-based
memory architecture, and (3) an effective automated curriculum that prioritises
tasks at the frontier of an agent's capabilities. We demonstrate characteristic
scaling laws with respect to network size, memory length, and richness of the
training task distribution. We believe our results lay the foundation for
increasingly general and adaptive RL agents that perform well across
ever-larger open-ended domains.

本文介绍了一种训练 RL 代理器的方法，通过使用 meta-reinforcement 学习、基于注意力的内存结构和有效的自动课程表，该代理器可以快速适应开放式的 3D 问题，并具有自适应性。