Existing work on VQA explores data augmentation to achieve better
generalization by perturbing the images in the dataset or modifying the
existing questions and answers. While these methods exhibit good performance,
the diversity of the questions and answers are constrained by the available
image set. In this work we explore using synthetic computer-generated data to
fully control the visual and language space, allowing us to provide more
diverse scenarios. We quantify the effect of synthetic data in real-world VQA
benchmarks and to which extent it produces results that generalize to real
data. By exploiting 3D and physics simulation platforms, we provide a pipeline
to generate synthetic data to expand and replace type-specific questions and
answers without risking the exposure of sensitive or personal data that might
be present in real images. We offer a comprehensive analysis while expanding
existing hyper-realistic datasets to be used for VQA. We also propose Feature
Swapping (F-SWAP) -- where we randomly switch object-level features during
training to make a VQA model more domain invariant. We show that F-SWAP is
effective for enhancing a currently existing VQA dataset of real images without
compromising on the accuracy to answer existing questions in the dataset.

采用合成数据生成技术，利用 3D 仿真平台生成了具备多样性的数据，针对现实世界 VQA 基准测试，量化了合成数据的影响并提出了 F-SWAP 方法，以提高现有真实图像数据集的 VQA 准确性。

SimVQA: 探索模拟环境进行视觉问答

SimVQA: Exploring Simulated Environments for Visual Question Answering

We present Habitat, a platform for research in embodied artificial
intelligence (AI). Habitat enables training embodied agents (virtual robots) in
highly efficient photorealistic 3D simulation. Specifically, Habitat consists
of: (i) Habitat-Sim: a flexible, high-performance 3D simulator with
configurable agents, sensors, and generic 3D dataset handling. Habitat-Sim is
fast -- when rendering a scene from Matterport3D, it achieves several thousand
frames per second (fps) running single-threaded, and can reach over 10,000 fps
multi-process on a single GPU. (ii) Habitat-API: a modular high-level library
for end-to-end development of embodied AI algorithms -- defining tasks (e.g.,
navigation, instruction following, question answering), configuring, training,
and benchmarking embodied agents.
These large-scale engineering contributions enable us to answer scientific
questions requiring experiments that were till now impracticable or 'merely'
impractical. Specifically, in the context of point-goal navigation: (1) we
revisit the comparison between learning and SLAM approaches from two recent
works and find evidence for the opposite conclusion -- that learning
outperforms SLAM if scaled to an order of magnitude more experience than
previous investigations, and (2) we conduct the first cross-dataset
generalization experiments {train, test} x {Matterport3D, Gibson} for multiple
sensors {blind, RGB, RGBD, D} and find that only agents with depth (D) sensors
generalize across datasets. We hope that our open-source platform and these
findings will advance research in embodied AI.

介绍 Habitat 平台，该平台在实现高效的真实感三维模拟环境中培训虚拟机器人及相关任务如导航和嵌入式人工智能方法等方面做出了重大工程贡献。通过使用 Habitat 平台，作者在点 - 目标导航方面，通过比较学习与 SLAM 方法，发现当学习处理的数据量超出先前调查的数量级时，学习方法比 SLAM 方法更优。此外，在进行多个数据集和传感器的横向通用性实验时，作者发现仅使用深度 (D) 传感器的模型才具有数据集通用性。作者希望 Habitat 平台及其研究成果能够推动嵌入式人工智能领域的发展。