As data science applications gain adoption across industries, the tooling
landscape matures to facilitate the life cycle of such applications and provide
solutions to the challenges involved to boost the productivity of the people
involved. Reinforcement learning with agents in a 3D world could still face
challenges: the knowledge required to use a simulation software as well as the
utilization of a standalone simulation software in unattended training
pipelines.
In this paper we review tools and approaches to train reinforcement learning
agents for robots in 3D worlds with respect to the robot Robotino and argue
that the separation of the simulation environment for creators of virtual
worlds and the model development environment for data scientists is not a well
covered topic. Often both are the same and data scientists require knowledge of
the simulation software to work directly with their APIs. Moreover, sometimes
creators of virtual worlds and data scientists even work on the same files. We
want to contribute to that topic by describing an approach where data
scientists don't require knowledge about the simulation software. Our approach
uses the standalone simulation software Webots, the Robot Operating System to
communicate with simulated robots as well as the simulation software itself and
container technology to separate the simulation from the model development
environment. We put emphasize on the APIs the data scientists work with and the
use of a standalone simulation software in unattended training pipelines. We
show the parts that are specific to the Robotino and the robot task to learn.

通过分离仿真环境和模型开发环境，并使用独立的仿真软件 Webots、机器人操作系统以及容器技术，我们描述了一种无需数据科学家了解仿真软件的方法，重点关注数据科学家所使用的 API 以及在无人值守训练管道中使用独立的仿真软件。

一个用于无人值守容器化（深层）强化学习的 Webots 架构

An Architecture for Unattended Containerized (Deep) Reinforcement  Learning with Webots

Despite impressive successes, deep reinforcement learning (RL) systems still
fall short of human performance on generalization to new tasks and environments
that differ from their training. As a benchmark tailored for studying RL
generalization, we introduce Avalon, a set of tasks in which embodied agents in
highly diverse procedural 3D worlds must survive by navigating terrain, hunting
or gathering food, and avoiding hazards. Avalon is unique among existing RL
benchmarks in that the reward function, world dynamics, and action space are
the same for every task, with tasks differentiated solely by altering the
environment; its 20 tasks, ranging in complexity from eat and throw to hunt and
navigate, each create worlds in which the agent must perform specific skills in
order to survive. This setup enables investigations of generalization within
tasks, between tasks, and to compositional tasks that require combining skills
learned from previous tasks. Avalon includes a highly efficient simulator, a
library of baselines, and a benchmark with scoring metrics evaluated against
hundreds of hours of human performance, all of which are open-source and
publicly available. We find that standard RL baselines make progress on most
tasks but are still far from human performance, suggesting Avalon is
challenging enough to advance the quest for generalizable RL.

通过引入针对强化学习通用化的 Avalon 挑战集，旨在帮助深度强化学习系统更好地适应于新任务和不同的环境，该集合基于高度多样化的 3D 环境，要求机器人体现出导航、狩猎和采集等能力，在每个地图中生存下来，该挑战集不仅限于改变环境仍使用相同的奖励函数、世界动力学和动作空间，每个任务都要求机器人在复杂程度上有所提升，Avalon 挑战集包括高效的仿真器和基准库，可用于基本检测和评分，现有的标准强化学习基线在大多数任务上都有进展，但仍远非人类表现，说明 Avalon 挑战集足够具有挑战性，可进一步推动深度强化学习通用化研究的进展。

Avalon: 使用程序生成世界的强化学习泛化基准

Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worlds

We introduce Amazon Berkeley Objects (ABO), a new large-scale dataset
designed to help bridge the gap between real and virtual 3D worlds. ABO
contains product catalog images, metadata, and artist-created 3D models with
complex geometries and physically-based materials that correspond to real,
household objects. We derive challenging benchmarks that exploit the unique
properties of ABO and measure the current limits of the state-of-the-art on
three open problems for real-world 3D object understanding: single-view 3D
reconstruction, material estimation, and cross-domain multi-view object
retrieval.

介绍 Amazon Berkeley Objects（ABO）数据集，旨在填补真实和虚拟 3D 世界之间的差距，并衡量了当前最先进技术在单视角 3D 重建、材料估计和跨域多视角物体检索等三个问题上的限制。