In this work we examine the use of Large Language Models (LLMs) in the
challenging setting of acting as a Minecraft agent. We apply and evaluate LLMs
in the builder and architect settings, introduce clarification questions and
examining the challenges and opportunities for improvement. In addition, we
present a platform for online interaction with the agents and an evaluation
against previous works.

本研究探讨在 Minecraft 代理设置中应用和评估大型语言模型（LLMs），介绍澄清问题并研究改进的挑战和机会，同时提供了与代理互动的在线平台，并与先前的工作进行了评估。

大型语言模型作为 Minecraft 代理

Large Language Models as Minecraft Agents

To pursue the goal of creating an open-ended agent in Minecraft, an
open-ended game environment with unlimited possibilities, this paper introduces
a task-centric framework named MCU for Minecraft agent evaluation. The MCU
framework leverages the concept of atom tasks as fundamental building blocks,
enabling the generation of diverse or even arbitrary tasks. Within the MCU
framework, each task is measured with six distinct difficulty scores (time
consumption, operational effort, planning complexity, intricacy, creativity,
novelty). These scores offer a multi-dimensional assessment of a task from
different angles, and thus can reveal an agent's capability on specific facets.
The difficulty scores also serve as the feature of each task, which creates a
meaningful task space and unveils the relationship between tasks. For efficient
evaluation of Minecraft agents employing the MCU framework, we maintain a
unified benchmark, namely SkillForge, which comprises representative tasks with
diverse categories and difficulty distribution. We also provide convenient
filters for users to select tasks to assess specific capabilities of agents. We
show that MCU has the high expressivity to cover all tasks used in recent
literature on Minecraft agent, and underscores the need for advancements in
areas such as creativity, precise control, and out-of-distribution
generalization under the goal of open-ended Minecraft agent development.

为了实现在 Minecraft 中创建一个开放式智能体的目标，本文介绍了一种名为 MCU 的以任务为中心的框架，该框架利用原子任务作为基本组成部分，能够生成多样甚至任意任务。MCU 框架通过六个不同的难度评分来衡量每个任务（时间消耗、操作工作量、规划复杂度、复杂性、创造力、新颖性），从不同角度对任务进行多维度评估，可以揭示智能体在特定方面的能力。任务的难度评分也作为每个任务的特征，从而创建了一个有意义的任务空间并揭示了任务之间的关系。为了高效评估使用 MCU 框架的 Minecraft 智能体，我们维护了一个统一的基准，名为 SkillForge，其中包括具有不同类别和难度分布的代表性任务，并为用户提供了方便的筛选器，以评估智能体的特定能力。我们展示了 MCU 具有高表达性，能够涵盖最近文献中使用的 Minecraft 智能体任务，并强调了在开发面向开放式 Minecraft 智能体的目标下在创造力、精确控制和超出分布的普遍化等领域的需求的重要性。