Language-and-vision navigation and question answering (QA) are exciting AI tasks situated at the intersection of natural language understanding, computer vision, and robotics. Researchers from all of these fields have begun creating datasets and model architectures for these domains. It is, however, not always clear if strong performance is due to advances in multimodal reasoning or if models are learning to exploit biases and artifacts of the data. We present single modality models and explore the linguistic, visual, and structural biases of these benchmarks. We find that single modality models often outperform published baselines that accompany multimodal task datasets, suggesting a need for change in community best practices moving forward. In light of this, we recommend presenting single modality baselines alongside new multimodal models to provide a fair comparison of information gained over dataset biases when considering multimodal input.

在多模态领域中，我们展示了单模态基线的惊人优势，并对未来研究提出了具体建议。我们认为，与随机和多数类基线相比，单模态方法更能捕捉和反映数据集的偏差，因此在评估多模态技术性能时提供了重要的比较。我们在三个最近的视觉导航和问答数据集上进行了单模态消融实验，发现与已发表的基线相比，性能增益高达29%的绝对值。

转移基准线：视觉导航与问答中的单模态性能