We introduce Replay, a collection of multi-view, multi-modal videos of humans
interacting socially. Each scene is filmed in high production quality, from
different viewpoints with several static cameras, as well as wearable action
cameras, and recorded with a large array of microphones at different positions
in the room. Overall, the dataset contains over 4000 minutes of footage and
over 7 million timestamped high-resolution frames annotated with camera poses
and partially with foreground masks. The Replay dataset has many potential
applications, such as novel-view synthesis, 3D reconstruction, novel-view
acoustic synthesis, human body and face analysis, and training generative
models. We provide a benchmark for training and evaluating novel-view
synthesis, with two scenarios of different difficulty. Finally, we evaluate
several baseline state-of-the-art methods on the new benchmark.

Replay 数据集是一组多视角、多模态的人类社交互动视频，可用于新视角合成、3D 重建、新视角声学合成、人体和面部分析以及生成模型训练，并提供了一个新视角合成的基准测试。