Recently emerged Vision-and-Language Navigation (VLN) tasks have drawn
significant attention in both computer vision and natural language processing
communities. Existing VLN tasks are built for agents that navigate on the
ground, either indoors or outdoors. However, many tasks require intelligent
agents to carry out in the sky, such as UAV-based goods delivery,
traffic/security patrol, and scenery tour, to name a few. Navigating in the sky
is more complicated than on the ground because agents need to consider the
flying height and more complex spatial relationship reasoning. To fill this gap
and facilitate research in this field, we propose a new task named AerialVLN,
which is UAV-based and towards outdoor environments. We develop a 3D simulator
rendered by near-realistic pictures of 25 city-level scenarios. Our simulator
supports continuous navigation, environment extension and configuration. We
also proposed an extended baseline model based on the widely-used
cross-modal-alignment (CMA) navigation methods. We find that there is still a
significant gap between the baseline model and human performance, which
suggests AerialVLN is a new challenging task. Dataset and code is available at
this https URL

提出了一项名为 AerialVLN 的新任务，基于无人机（UAV）的、面向室外环境的导航，在提出的 3D 模拟器中使用高度逼真的城市景观照片进行连续导航并进行环境扩展和配置，基于跨模态对齐（CMA）导航方法的扩展基线模型，发现基线模型与人类表现之间仍存在显著差距，表明 AerialVLN 是一项具有挑战性的新任务。

无人机视觉与语言导航：AerialVLN

AerialVLN: Vision-and-Language Navigation for UAVs

Given a natural language, a general robot has to comprehend the instruction
and find the target object or location based on visual observations even in
unexplored environments. Most agents rely on massive diverse training data to
achieve better generalization, which requires expensive labor. These agents
often focus on common objects and fewer tasks, thus are not intelligent enough
to handle different types of instructions. To facilitate research in open-set
vision-and-language navigation, we propose a benchmark named MO-VLN, aiming at
testing the effectiveness and generalization of the agent in the multi-task
setting. First, we develop a 3D simulator rendered by realistic scenarios using
Unreal Engine 5, containing more realistic lights and details. The simulator
contains three scenes, i.e., cafe, restaurant, and nursing house, of high value
in the industry. Besides, our simulator involves multiple uncommon objects,
such as takeaway cup and medical adhesive tape, which are more complicated
compared with existing environments. Inspired by the recent success of large
language models (e.g., ChatGPT, Vicuna), we construct diverse high-quality data
of instruction type without human annotation. Our benchmark MO-VLN provides
four tasks: 1) goal-conditioned navigation given a specific object category
(e.g., "fork"); 2) goal-conditioned navigation given simple instructions (e.g.,
"Search for and move towards a tennis ball"); 3) step-by-step instruction
following; 4) finding abstract object based on high-level instruction (e.g., "I
am thirsty").

提出一个名为 MO-VLN 的基准测试系统，用于测试机器人智能导航的效果和泛化能力。该系统包含三个逼真的 3D 场景，并涉及多个复杂的非常见物体，任务包括目标定位和高级指令追踪等。

MO-VLN: 一个用于开放集零样本视觉与语言导航的多任务基准测试

MO-VLN: A Multi-Task Benchmark for Open-set Zero-Shot  Vision-and-Language Navigation

This paper studies the problem of predicting the distribution over multiple
possible future paths of people as they move through various visual scenes. We
make two main contributions. The first contribution is a new dataset, created
in a realistic 3D simulator, which is based on real world trajectory data, and
then extrapolated by human annotators to achieve different latent goals. This
provides the first benchmark for quantitative evaluation of the models to
predict multi-future trajectories. The second contribution is a new model to
generate multiple plausible future trajectories, which contains novel designs
of using multi-scale location encodings and convolutional RNNs over graphs. We
refer to our model as Multiverse. We show that our model achieves the best
results on our dataset, as well as on the real-world VIRAT/ActEV dataset (which
just contains one possible future).

本文提出了一个名为 Multiverse 的新模型来生成多个合理的未来人物运动路径，并使用 3D 模拟器创建了一个名为 VIRAT/ActEV 的数据集用以对该模型进行测试。通过使用多尺度位置编码和图卷积神经网络，该模型达到了最佳效果。

多重未来轨迹预测之分岔路径园

The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction

Recent progress in the field of reinforcement learning has been accelerated
by virtual learning environments such as video games, where novel algorithms
and ideas can be quickly tested in a safe and reproducible manner. We introduce
the Google Research Football Environment, a new reinforcement learning
environment where agents are trained to play football in an advanced,
physics-based 3D simulator. The resulting environment is challenging, easy to
use and customize, and it is available under a permissive open-source license.
In addition, it provides support for multiplayer and multi-agent experiments.
We propose three full-game scenarios of varying difficulty with the Football
Benchmarks and report baseline results for three commonly used reinforcement
algorithms (IMPALA, PPO, and Ape-X DQN). We also provide a diverse set of
simpler scenarios with the Football Academy and showcase several promising
research directions.

介绍了一项新的强化学习环境 - 谷歌足球环境，提供支持多人和多代理实验的虚拟学习环境和用于测试和展示强化学习算法的三种难度场景与一组更简单的场景。