We introduce Nymeria - a large-scale, diverse, richly annotated human motion
dataset collected in the wild with multiple multimodal egocentric devices. The
dataset comes with a) full-body 3D motion ground truth; b) egocentric
multimodal recordings from Project Aria devices with RGB, grayscale,
eye-tracking cameras, IMUs, magnetometer, barometer, and microphones; and c) an
additional "observer" device providing a third-person viewpoint. We compute
world-aligned 6DoF transformations for all sensors, across devices and capture
sessions. The dataset also provides 3D scene point clouds and calibrated gaze
estimation. We derive a protocol to annotate hierarchical language descriptions
of in-context human motion, from fine-grain pose narrations, to atomic actions
and activity summarization. To the best of our knowledge, the Nymeria dataset
is the world largest in-the-wild collection of human motion with natural and
diverse activities; first of its kind to provide synchronized and localized
multi-device multimodal egocentric data; and the world largest dataset with
motion-language descriptions. It contains 1200 recordings of 300 hours of daily
activities from 264 participants across 50 locations, travelling a total of
399Km. The motion-language descriptions provide 310.5K sentences in 8.64M words
from a vocabulary size of 6545. To demonstrate the potential of the dataset we
define key research tasks for egocentric body tracking, motion synthesis, and
action recognition and evaluate several state-of-the-art baseline algorithms.
Data and code will be open-sourced.

介绍了一个大规模的人体运动数据集 Nymeria，在野外使用多个多模态视角设备收集。数据集具有全身三维运动真实值、多模态视角设备的记录以及附加的第三人称视角设备。数据集还提供了三维场景点云和校准的注视估计。定义了几个关于自我中心身体追踪、动作合成和动作识别的关键研究任务，并评估了几种最先进的基准算法。