Probabilistic world models increase data efficiency of model-based
reinforcement learning (MBRL) by guiding the policy with their epistemic
uncertainty to improve exploration and acquire new samples. Moreover, the
uncertainty-aware learning procedures in probabilistic approaches lead to
robust policies that are less sensitive to noisy observations compared to
uncertainty unaware solutions. We propose to combine trajectory sampling and
deep Gaussian covariance network (DGCN) for a data-efficient solution to MBRL
problems in an optimal control setting. We compare trajectory sampling with
density-based approximation for uncertainty propagation using three different
probabilistic world models; Gaussian processes, Bayesian neural networks, and
DGCNs. We provide empirical evidence using four different well-known test
environments, that our method improves the sample-efficiency over other
combinations of uncertainty propagation methods and probabilistic models.
During our tests, we place particular emphasis on the robustness of the learned
policies with respect to noisy initial states.

通过结合轨迹采样和深度高斯协方差网络（DGCN），我们提出了一种在最优控制环境中解决 MBRL 问题的数据高效解决方案，通过三种不同的概率世界模型（高斯过程、贝叶斯神经网络和 DGCNs）比较采用基于密度的不确定性传播的轨迹采样方法，我们在四个不同的知名测试环境中提供了实证证据，表明我们的方法提高了样本效率，同时对于噪声初始状态的鲁棒性也得到了重视。

采样轨迹的深度高斯协方差网络用于数据有效的策略搜索

Deep Gaussian Covariance Network with Trajectory Sampling for  Data-Efficient Policy Search

Installing probabilistic world models into artificial agents opens an
efficient channel for humans to communicate with and control these agents. In
addition to updating agent policies, humans can modify their internal world
models in order to influence their decisions. The challenge, however, is that
currently existing world models are difficult for humans to adapt because they
lack a natural communication interface. Aimed at addressing this shortcoming,
we develop Language-Guided World Models (LWMs), which can capture environment
dynamics by reading language descriptions. These models enhance agent
communication efficiency, allowing humans to simultaneously alter their
behavior on multiple tasks with concise language feedback. They also enable
agents to self-learn from texts originally written to instruct humans. To
facilitate the development of LWMs, we design a challenging benchmark based on
the game of MESSENGER (Hanjie et al., 2021), requiring compositional
generalization to new language descriptions and environment dynamics. Our
experiments reveal that the current state-of-the-art Transformer architecture
performs poorly on this benchmark, motivating us to design a more robust
architecture. To showcase the practicality of our proposed LWMs, we simulate a
scenario where these models augment the interpretability and safety of an agent
by enabling it to generate and discuss plans with a human before execution. By
effectively incorporating language feedback on the plan, the models boost the
agent performance in the real environment by up to three times without
collecting any interactive experiences in this environment.

安装概率世界模型到人工智能代理中，为人类与控制这些代理打开了一个高效的交流途径；我们开发了一种名为语言引导的世界模型（LWMs），通过阅读语言描述来捕捉环境动态，提高了代理的通信效率，同时允许人类用简洁的语言反馈在多个任务中同时改变行为。该研究证明了当前最先进的 Transformer 架构在该基准测试上表现不佳，激励我们设计更强大的架构。通过模拟展示了我们提出的 LWMs 的实用性，使代理能够在执行前生成和讨论计划，增强了代理的可解释性和安全性，并使其在真实环境中的性能提高了三倍，而无需在该环境中进行任何交互式经验的收集。

语言引导的世界模型：一种基于模型的人工智能控制方法

Language-Guided World Models: A Model-Based Approach to AI Control

For autonomous agents to act as trustworthy partners to human users, they
must be able to reliably communicate their competency for the tasks they are
asked to perform. Towards this objective, we develop probabilistic world models
based on deep generative modelling that allow for the simulation of agent
trajectories and accurate calculation of tasking outcome probabilities. By
combining the strengths of conditional variational autoencoders with recurrent
neural networks, the deep generative world model can probabilistically forecast
trajectories over long horizons to task completion. We show how these
forecasted trajectories can be used to calculate outcome probability
distributions, which enable the precise assessment of agent competency for
specific tasks and initial settings.

文章介绍了一种基于深度生成建模的概率世界模型，使得自主代理机器人能够可靠地沟通其在所需任务中的能力并且可以计算出任务完成的精确概率分布。