In reinforcement learning for visual navigation, it is common to develop a
model for each new task, and train that model from scratch with task-specific
interactions in 3D environments. However, this process is expensive; massive
amounts of interactions are needed for the model to generalize well. Moreover,
this process is repeated whenever there is a change in the task type or the
goal modality. We present a unified approach to visual navigation using a novel
modular transfer learning model. Our model can effectively leverage its
experience from one source task and apply it to multiple target tasks (e.g.,
ObjectNav, RoomNav, ViewNav) with various goal modalities (e.g., image, sketch,
audio, label). Furthermore, our model enables zero-shot experience learning,
whereby it can solve the target tasks without receiving any task-specific
interactive training. Our experiments on multiple photorealistic datasets and
challenging tasks show that our approach learns faster, generalizes better, and
outperforms SoTA models by a significant margin.

本研究提出了一种采用新型模块化迁移学习模型的视觉导航统一方法。该模型可以有效地利用从一个源任务积累的经验并将其应用于多个目标任务（例如，ObjectNav、RoomNav、ViewNav）以及具有各种目标模态（例如，图像、草图、音频、标签）的目标任务，同时实现了零点经验学习，这让模型可以在不接收任何任务特定的交互式训练的情况下解决目标任务。实验结果表明，与现有最优算法相比，我们的方法可以更快地学习，实现更好的泛化，并获得了明显的性能优势。

零经验要求：语义视觉导航的即插即用模块化迁移学习

Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation

We propose a novel framework for creating large-scale photorealistic datasets
of indoor scenes, with ground truth geometry, material, lighting and semantics.
Our goal is to make the dataset creation process widely accessible,
transforming scans into photorealistic datasets with high-quality ground truth
for appearance, layout, semantic labels, high quality spatially-varying BRDF
and complex lighting, including direct, indirect and visibility components.
This enables important applications in inverse rendering, scene understanding
and robotics. We show that deep networks trained on the proposed dataset
achieve competitive performance for shape, material and lighting estimation on
real images, enabling photorealistic augmented reality applications, such as
object insertion and material editing. We also show our semantic labels may be
used for segmentation and multi-task learning. Finally, we demonstrate that our
framework may also be integrated with physics engines, to create virtual
robotics environments with unique ground truth such as friction coefficients
and correspondence to real scenes. The dataset and all the tools to create such
datasets will be made publicly available.

提出了一种创造大规模室内场景的照片级真实数据集的新框架，可提供外观、布局、语义标签、高质量变化自由的 BRDF 和复杂照明等显著真实的基准数据，可用于逆向渲染、场景理解和机器人学，与物理引擎结合可创建具有摩擦系数等真实场景对应关系的虚拟机器人环境。