Neural rendering is fuelling a unification of learning, 3D geometry and video
understanding that has been waiting for more than two decades. Progress,
however, is still hampered by a lack of suitable datasets and benchmarks. To
address this gap, we introduce EPIC Fields, an augmentation of EPIC-KITCHENS
with 3D camera information. Like other datasets for neural rendering, EPIC
Fields removes the complex and expensive step of reconstructing cameras using
photogrammetry, and allows researchers to focus on modelling problems. We
illustrate the challenge of photogrammetry in egocentric videos of dynamic
actions and propose innovations to address them. Compared to other neural
rendering datasets, EPIC Fields is better tailored to video understanding
because it is paired with labelled action segments and the recent VISOR segment
annotations. To further motivate the community, we also evaluate two benchmark
tasks in neural rendering and segmenting dynamic objects, with strong baselines
that showcase what is not possible today. We also highlight the advantage of
geometry in semi-supervised video object segmentations on the VISOR
annotations. EPIC Fields reconstructs 96% of videos in EPICKITCHENS,
registering 19M frames in 99 hours recorded in 45 kitchens.

介绍了一个名为 EPIC Fields 的神经渲染数据集，能够通过附加 3D 相机信息，为图像和视频理解提供更好的数据和基准，同时提供了两个神经渲染和分割动态对象的基准任务。