We present a new dataset called Real Acoustic Fields (RAF) that captures real
acoustic room data from multiple modalities. The dataset includes high-quality
and densely captured room impulse response data paired with multi-view images,
and precise 6DoF pose tracking data for sound emitters and listeners in the
rooms. We used this dataset to evaluate existing methods for novel-view
acoustic synthesis and impulse response generation which previously relied on
synthetic data. In our evaluation, we thoroughly assessed existing audio and
audio-visual models against multiple criteria and proposed settings to enhance
their performance on real-world data. We also conducted experiments to
investigate the impact of incorporating visual data (i.e., images and depth)
into neural acoustic field models. Additionally, we demonstrated the
effectiveness of a simple sim2real approach, where a model is pre-trained with
simulated data and fine-tuned with sparse real-world data, resulting in
significant improvements in the few-shot learning approach. RAF is the first
dataset to provide densely captured room acoustic data, making it an ideal
resource for researchers working on audio and audio-visual neural acoustic
field modeling techniques. Demos and datasets are available on our project
page: this https URL

我们提出了一个名为 Real Acoustic Fields（RAF）的新数据集，其中包含了从多种模态捕获的真实声场数据。该数据集包括了与多视图图像配对的高质量密集捕获的房间脉冲响应数据，以及声音发射器和听者的精确 6DoF 姿态跟踪数据。我们利用该数据集评估了现有的新视角声学合成和脉冲响应生成方法，并提出了用于增强它们在真实世界数据上表现的设置。我们还通过实验调查了将视觉数据（图像和深度）与神经声学场模型相结合的影响。此外，我们展示了一种简单的模拟 - 真实方法的有效性，即使用模拟数据进行预训练，并用稀疏的真实数据进行微调，从而显著改善了少样本学习方法。RAF 是第一个提供了密集捕获房间声学数据的数据集，对于从事音频和音频 - 视觉神经声学场建模技术的研究人员来说，它是一个理想的资源。我们的项目页面提供演示和数据集：https:// 此处 URL