Meta-reinforcement learning enables artificial agents to learn from related
training tasks and adapt to new tasks efficiently with minimal interaction
data. However, most existing research is still limited to narrow task
distributions that are parametric and stationary, and does not consider
out-of-distribution tasks during the evaluation, thus, restricting its
application. In this paper, we propose MoSS, a context-based Meta-reinforcement
learning algorithm based on Self-Supervised task representation learning to
address this challenge. We extend meta-RL to broad non-parametric task
distributions which have never been explored before, and also achieve
state-of-the-art results in non-stationary and out-of-distribution tasks.
Specifically, MoSS consists of a task inference module and a policy module. We
utilize the Gaussian mixture model for task representation to imitate the
parametric and non-parametric task variations. Additionally, our online
adaptation strategy enables the agent to react at the first sight of a task
change, thus being applicable in non-stationary tasks. MoSS also exhibits
strong generalization robustness in out-of-distributions tasks which benefits
from the reliable and robust task representation. The policy is built on top of
an off-policy RL algorithm and the entire network is trained completely
off-policy to ensure high sample efficiency. On MuJoCo and Meta-World
benchmarks, MoSS outperforms prior works in terms of asymptotic performance,
sample efficiency (3-50x faster), adaptation efficiency, and generalization
robustness on broad and diverse task distributions.

本文提出了 MoSS 算法，它是基于上下文自监督任务表示学习的元强化学习算法，能够在非参数任务分布、非静态任务和分布任务等方面实现先进的数据效率和适应能力。