Behavioral cloning uses a dataset of demonstrations to learn a policy. To
overcome computationally expensive training procedures and address the policy
adaptation problem, we propose to use latent spaces of pre-trained foundation
models to index a demonstration dataset, instantly access similar relevant
experiences, and copy behavior from these situations. Actions from a selected
similar situation can be performed by the agent until representations of the
agent's current situation and the selected experience diverge in the latent
space. Thus, we formulate our control problem as a dynamic search problem over
a dataset of experts' demonstrations. We test our approach on BASALT
MineRL-dataset in the latent representation of a Video Pre-Training model. We
compare our model to state-of-the-art, Imitation Learning-based Minecraft
agents. Our approach can effectively recover meaningful demonstrations and show
human-like behavior of an agent in the Minecraft environment in a wide variety
of scenarios. Experimental results reveal that performance of our search-based
approach clearly wins in terms of accuracy and perceptual evaluation over
learning-based models.

使用预训练的基础模型的潜在空间索引演示数据集，通过复制类似情境中的行为来解决具有计算成本的训练过程和策略适应问题，实验结果显示该方法在准确性和知觉评估方面明显优于基于学习的模型，能在 Minecraft 环境中以人类样式表现出智能行为。