Sim2real transfer has received increasing attention lately due to the success
of learning robotic tasks in simulation end-to-end. While there has been a lot
of progress in transferring vision-based navigation policies, the existing
sim2real strategy for audio-visual navigation performs data augmentation
empirically without measuring the acoustic gap. The sound differs from light in
that it spans across much wider frequencies and thus requires a different
solution for sim2real. We propose the first treatment of sim2real for
audio-visual navigation by disentangling it into acoustic field prediction
(AFP) and waypoint navigation. We first validate our design choice in the
SoundSpaces simulator and show improvement on the Continuous AudioGoal
navigation benchmark. We then collect real-world data to measure the spectral
difference between the simulation and the real world by training AFP models
that only take a specific frequency subband as input. We further propose a
frequency-adaptive strategy that intelligently selects the best frequency band
for prediction based on both the measured spectral difference and the energy
distribution of the received audio, which improves the performance on the real
data. Lastly, we build a real robot platform and show that the transferred
policy can successfully navigate to sounding objects. This work demonstrates
the potential of building intelligent agents that can see, hear, and act
entirely from simulation, and transferring them to the real world.

在学习机器人任务与模拟端到端成功的基础上，视觉导航策略的转换取得了很大进展，然而，现有的声音 - 视觉导航的模拟到真实转换策略在进行数据增强时往往是经验性的，而没有对声学差异进行测量。本文提出了首个声音 - 视觉导航的模拟到真实处理方法，将其分解为声学场预测和路标导航，通过在 SoundSpaces 模拟器上验证我们的设计选择并在连续音频目标导航基准上展示改进，通过训练仅接受特定频率子带作为输入的声学场预测模型来收集真实世界数据，测量模拟与真实世界之间的频谱差异，进一步提出基于测量频谱差异和接收音频能量分布的频率自适应策略，从而改善在真实数据上的性能，最后，我们构建了一个真实的机器人平台，并展示了转化的策略能够成功导航到具有声音的对象。这项工作展示了在完全从模拟中看、听和行动的智能代理构建的潜力，并将其转移到真实世界。