We introduce Nymeria - a large-scale, diverse, richly annotated human motion
dataset collected in the wild with multiple multimodal egocentric devices. The
dataset comes with a) full-body 3D motion ground truth; b) egocentric
multimodal recordings from Project Aria devices with RGB, grayscale,
eye-tracking cameras, IMUs, magnetometer, barometer, and microphones; and c) an
additional "observer" device providing a third-person viewpoint. We compute
world-aligned 6DoF transformations for all sensors, across devices and capture
sessions. The dataset also provides 3D scene point clouds and calibrated gaze
estimation. We derive a protocol to annotate hierarchical language descriptions
of in-context human motion, from fine-grain pose narrations, to atomic actions
and activity summarization. To the best of our knowledge, the Nymeria dataset
is the world largest in-the-wild collection of human motion with natural and
diverse activities; first of its kind to provide synchronized and localized
multi-device multimodal egocentric data; and the world largest dataset with
motion-language descriptions. It contains 1200 recordings of 300 hours of daily
activities from 264 participants across 50 locations, travelling a total of
399Km. The motion-language descriptions provide 310.5K sentences in 8.64M words
from a vocabulary size of 6545. To demonstrate the potential of the dataset we
define key research tasks for egocentric body tracking, motion synthesis, and
action recognition and evaluate several state-of-the-art baseline algorithms.
Data and code will be open-sourced.

介绍了一个大规模的人体运动数据集 Nymeria，在野外使用多个多模态视角设备收集。数据集具有全身三维运动真实值、多模态视角设备的记录以及附加的第三人称视角设备。数据集还提供了三维场景点云和校准的注视估计。定义了几个关于自我中心身体追踪、动作合成和动作识别的关键研究任务，并评估了几种最先进的基准算法。

Nymeria: 野外大规模多模态自我中心的日常动作数据集

Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in  the Wild

The study of biological materials and bio-inspired materials science is well
established; however, surprisingly little knowledge has been systematically
translated to engineering solutions. To accelerate discovery and guide
insights, an open-source autoregressive transformer large language model,
BioinspiredLLM, is reported. The model was finetuned with a corpus of over a
thousand peer-reviewed articles in the field of structural biological and
bio-inspired materials and can be prompted to actively and interactively recall
information, assist with research tasks, and function as an engine for
creativity. The model has proven by example that it is not only able to
accurately recall information about biological materials when queried but also
formulate biomaterials questions and answers that can evaluate its own
performance. BioinspiredLLM also has been shown to develop sound hypotheses
regarding biological materials design and remarkably so for materials that have
never been explicitly studied before. Lastly, the model showed impressive
promise in collaborating with other generative artificial intelligence models
in a workflow that can reshape the traditional materials design process. This
collaborative generative artificial intelligence method can stimulate and
enhance bio-inspired materials design workflows. Biological materials is at a
critical intersection of multiple scientific fields and models like
BioinspiredLLM help to connect knowledge domains.

通过自动回归变换器大型语言模型 BioinspiredLLM，我们可以加速发现并指导研究，通过该模型可以提供信息回忆、协助研究任务和推动创造力等功能，同时它还能与其他生成式人工智能模型协作，重塑传统材料设计流程。