Animals perceive the world to plan their actions and interact with other agents to accomplish complex tasks, demonstrating capabilities that are still unmatched by AI systems. To advance our understanding and reduce the gap between the capabilities of animals and AI systems, we introduce a dataset of pet egomotion imagery with diverse examples of simultaneous egomotion and multi-agent interaction. Current video datasets separately contain egomotion and interaction examples, but rarely both at the same time. In addition, EgoPet offers a radically distinct perspective from existing egocentric datasets of humans or vehicles. We define two in-domain benchmark tasks that capture animal behavior, and a third benchmark to assess the utility of EgoPet as a pretraining resource to robotic quadruped locomotion, showing that models trained from EgoPet outperform those trained from prior datasets.

为了推动我们对动物能力与人工智能系统能力之间的理解，我们引入了一个包含多样化的宠物自我运动图像和多智能体互动的数据集。该数据集与现有的人类或车辆自我中心数据集形成鲜明对比，定义了捕捉动物行为的两个基准任务，并通过与先前数据集相比，证明从EgoPet训练的模型表现更好，作为机器四足动作预训练资源的实用性。

EgoPet: 一种来自动物视角的自我运动和互动数据