Artificial Intelligence (AI) systems planned for deployment in real-world
applications frequently are researched and developed in closed simulation
environments where all variables are controlled and known to the simulator or
labeled benchmark datasets are used. Transition from these simulators,
testbeds, and benchmark datasets to more open-world domains poses significant
challenges to AI systems, including significant increases in the complexity of
the domain and the inclusion of real-world novelties; the open-world
environment contains numerous out-of-distribution elements that are not part in
the AI systems' training set. Here, we propose a path to a general,
domain-independent measure of domain complexity level. We distinguish two
aspects of domain complexity: intrinsic and extrinsic. The intrinsic domain
complexity is the complexity that exists by itself without any action or
interaction from an AI agent performing a task on that domain. This is an
agent-independent aspect of the domain complexity. The extrinsic domain
complexity is agent- and task-dependent. Intrinsic and extrinsic elements
combined capture the overall complexity of the domain. We frame the components
that define and impact domain complexity levels in a domain-independent light.
Domain-independent measures of complexity could enable quantitative predictions
of the difficulty posed to AI systems when transitioning from one testbed or
environment to another, when facing out-of-distribution data in open-world
tasks, and when navigating the rapidly expanding solution and search spaces
encountered in open-world domains.

该研究提出了一种通向一般、领域无关的领域复杂度水平的路径，该水平由固有和外在两个要素组成，可以帮助量化预测人工智能系统在转变测试环境、面对开放世界任务中的分布数据以及导航在开放世界领域中快速扩展的解决方案和搜索空间时所面临的困难。

跨领域定义领域复杂度度量的探索

Toward Defining a Domain Complexity Measure Across Domains

We study open-world 3D scene understanding, a family of tasks that require
agents to reason about their 3D environment with an open-set vocabulary and
out-of-domain visual inputs - a critical skill for robots to operate in the
unstructured 3D world. Towards this end, we propose Semantic Abstraction
(SemAbs), a framework that equips 2D Vision-Language Models (VLMs) with new 3D
spatial capabilities, while maintaining their zero-shot robustness. We achieve
this abstraction using relevancy maps extracted from CLIP, and learn 3D spatial
and geometric reasoning skills on top of those abstractions in a
semantic-agnostic manner. We demonstrate the usefulness of SemAbs on two
open-world 3D scene understanding tasks: 1) completing partially observed
objects and 2) localizing hidden objects from language descriptions.
Experiments show that SemAbs can generalize to novel vocabulary,
materials/lighting, classes, and domains (i.e., real-world scans) from training
on limited 3D synthetic data. Code and data is available at
this https URL

本文介绍了一种基于视觉语言模型的语义抽象框架，可以为机器人在未结构化的三维环境中推理，并学会三维空间和几何推理技能，并在两个开放式三维场景理解任务上进行了实验测试。

语义抽象：从 2D 视觉语言模型实现开放式三维场景理解

Semantic Abstraction: Open-World 3D Scene Understanding from 2D  Vision-Language Models

We held the first-ever MineRL Benchmark for Agents that Solve Almost-Lifelike
Tasks (MineRL BASALT) Competition at the Thirty-fifth Conference on Neural
Information Processing Systems (NeurIPS 2021). The goal of the competition was
to promote research towards agents that use learning from human feedback (LfHF)
techniques to solve open-world tasks. Rather than mandating the use of LfHF
techniques, we described four tasks in natural language to be accomplished in
the video game Minecraft, and allowed participants to use any approach they
wanted to build agents that could accomplish the tasks. Teams developed a
diverse range of LfHF algorithms across a variety of possible human feedback
types. The three winning teams implemented significantly different approaches
while achieving similar performance. Interestingly, their approaches performed
well on different tasks, validating our choice of tasks to include in the
competition. While the outcomes validated the design of our competition, we did
not get as many participants and submissions as our sister competition, MineRL
Diamond. We speculate about the causes of this problem and suggest improvements
for future iterations of the competition.

本文介绍了 MineRL BASALT 比赛，旨在鼓励研究使用来自人类反馈的学习技巧的代理解决开放世界任务，参赛团队的多样化反映出任务的设计有效性，但比赛参与者不足，作者对此进行了探讨和提出了改善建议。