Augmented Reality (AR) as a platform has the potential to facilitate the
reduction of the cocktail party effect. Future AR headsets could potentially
leverage information from an array of sensors spanning many different
modalities. Training and testing signal processing and machine learning
algorithms on tasks such as beam-forming and speech enhancement require high
quality representative data. To the best of the author's knowledge, as of
publication there are no available datasets that contain synchronized
egocentric multi-channel audio and video with dynamic movement and
conversations in a noisy environment. In this work, we describe, evaluate and
release a dataset that contains over 5 hours of multi-modal data useful for
training and testing algorithms for the application of improving conversations
for an AR glasses wearer. We provide speech intelligibility, quality and
signal-to-noise ratio improvement results for a baseline method and show
improvements across all tested metrics. The dataset we are releasing contains
AR glasses egocentric multi-channel microphone array audio, wide field-of-view
RGB video, speech source pose, headset microphone audio, annotated voice
activity, speech transcriptions, head bounding boxes, target of speech and
source identification labels. We have created and are releasing this dataset to
facilitate research in multi-modal AR solutions to the cocktail party problem.

本文发现，缺少可用于训练和测试信号处理和机器学习算法的具有代表性的数据集，介绍并放出了一个包含超过 5 小时多模态数据的数据集，以便建立多模态 AR 方案以减少鸡尾酒会效应。