Large language models (LLMs) have achieved impressive progress on several
open-world tasks. Recently, using LLMs to build embodied agents has been a
hotspot. In this paper, we propose STEVE, a comprehensive and visionary
embodied agent in the Minecraft virtual environment. STEVE consists of three
key components: vision perception, language instruction, and code action.
Vision perception involves the interpretation of visual information in the
environment, which is then integrated into the LLMs component with agent state
and task instruction. Language instruction is responsible for iterative
reasoning and decomposing complex tasks into manageable guidelines. Code action
generates executable skill actions based on retrieval in skill database,
enabling the agent to interact effectively within the Minecraft environment. We
also collect STEVE-21K dataset, which includes 600$+$ vision-environment pairs,
20K knowledge question-answering pairs, and 200$+$ skill-code pairs. We conduct
continuous block search, knowledge question and answering, and tech tree
mastery to evaluate the performance. Extensive experiments show that STEVE
achieves at most $1.5 \times$ faster unlocking key tech trees and $2.5 \times$
quicker in block search tasks compared to previous state-of-the-art methods.

STEVE 是一个在 Minecraft 虚拟环境中基于大型语言模型的综合和具有远见的具象代理，其三个关键组成部分是视觉感知、语言指导和代码动作，通过视觉信息解释、迭代推理和可执行技能行为生成，STEVE 在 Minecraft 环境中实现更快的技能解锁和方块搜索任务。

目视与思考：虚拟环境中的具身智能体

See and Think: Embodied Agent in Virtual Environment

Unsupervised object-centric learning aims to represent the modular,
compositional, and causal structure of a scene as a set of object
representations and thereby promises to resolve many critical limitations of
traditional single-vector representations such as poor systematic
generalization. Although there have been many remarkable advances in recent
years, one of the most critical problems in this direction has been that
previous methods work only with simple and synthetic scenes but not with
complex and naturalistic images or videos. In this paper, we propose STEVE, an
unsupervised model for object-centric learning in videos. Our proposed model
makes a significant advancement by demonstrating its effectiveness on various
complex and naturalistic videos unprecedented in this line of research.
Interestingly, this is achieved by neither adding complexity to the model
architecture nor introducing a new objective or weak supervision. Rather, it is
achieved by a surprisingly simple architecture that uses a transformer-based
image decoder conditioned on slots and the learning objective is simply to
reconstruct the observation. Our experiment results on various complex and
naturalistic videos show significant improvements compared to the previous
state-of-the-art.

提出了 STEVE，一种基于无监督学习的视频物体聚焦模型，通过使用基于 Transformer 的图像解码器进行重建观察来实现了对复杂和自然视频的有效性验证，并取得了显著的改进效果。

面向复杂和自然视频的简单无监督物体中心学习

Simple Unsupervised Object-Centric Learning for Complex and Naturalistic  Videos

In this work we present STEVE - Soccer TEam VEctors, a principled approach
for learning real valued vectors for soccer teams where similar teams are close
to each other in the resulting vector space. STEVE only relies on freely
available information about the matches teams played in the past. These vectors
can serve as input to various machine learning tasks. Evaluating on the task of
team market value estimation, STEVE outperforms all its competitors. Moreover,
we use STEVE for similarity search and to rank soccer teams.

本文提出了 STEVE - 足球队向量的一种方法，该方法为学习足球队的实值向量，使得相似的球队在所得到的向量空间中彼此靠近。此方法仅依赖于有关球队过去比赛的免费信息，向量可用于各种机器学习任务中。在团队市场价值估计任务中，STEVE 的表现超过所有竞争对手。此外，我们使用 STEVE 进行相似性搜索和排名足球队。