The advent of wearable computers enables a new source of context for AI that
is embedded in egocentric sensor data. This new egocentric data comes equipped
with fine-grained 3D location information and thus presents the opportunity for
a novel class of spatial foundation models that are rooted in 3D space. To
measure progress on what we term Egocentric Foundation Models (EFMs) we
establish EFM3D, a benchmark with two core 3D egocentric perception tasks.
EFM3D is the first benchmark for 3D object detection and surface regression on
high quality annotated egocentric data of Project Aria. We propose Egocentric
Voxel Lifting (EVL), a baseline for 3D EFMs. EVL leverages all available
egocentric modalities and inherits foundational capabilities from 2D foundation
models. This model, trained on a large simulated dataset, outperforms existing
methods on the EFM3D benchmark.

佩戴型计算机的出现为 AI 提供了一种嵌入式于自我中心传感器数据中的新的上下文来源。本文建立了 EFM3D，一个针对 3D 自我中心感知任务的基准，并提出了 EVL 作为 EFM3D 的基线模型，在大规模模拟数据集上进行训练，并在 EFM3D 基准上优于现有方法。