Human perception of the world is shaped by a multitude of viewpoints and
modalities. While many existing datasets focus on scene understanding from a
certain perspective (e.g. egocentric or third-person views), our dataset offers
a panoptic perspective (i.e. multiple viewpoints with multiple data
modalities). Specifically, we encapsulate third-person panoramic and front
views, as well as egocentric monocular/binocular views with rich modalities
including video, multi-channel audio, directional binaural delay, location data
and textual scene descriptions within each scene captured, presenting
comprehensive observation of the world. Figure 1 offers a glimpse of all 28
scene categories of our 360+x dataset. To the best of our knowledge, this is
the first database that covers multiple viewpoints with multiple data
modalities to mimic how daily information is accessed in the real world.
Through our benchmark analysis, we presented 5 different scene understanding
tasks on the proposed 360+x dataset to evaluate the impact and benefit of each
data modality and perspective in panoptic scene understanding. We hope this
unique dataset could broaden the scope of comprehensive scene understanding and
encourage the community to approach these problems from more diverse
perspectives.

该研究使用多个视角和多种数据模式，提供全景视角和个人视角的场景理解数据集，以模拟真实世界中的信息访问方式，并通过 5 个不同的场景理解任务对数据集进行了评估，希望为综合场景理解提供更广泛的范围和多元化视角。