Large-scale pre-trained models (PTMs) show great zero-shot capabilities. In
this paper, we study how to leverage them for zero-shot visual question
answering (VQA). Our approach is motivated by a few observations. First, VQA
questions often require multiple steps of reasoning, which is still a
capability that most PTMs lack. Second, different steps in VQA reasoning chains
require different skills such as object detection and relational reasoning, but
a single PTM may not possess all these skills. Third, recent work on zero-shot
VQA does not explicitly consider multi-step reasoning chains, which makes them
less interpretable compared with a decomposition-based approach. We propose a
modularized zero-shot network that explicitly decomposes questions into sub
reasoning steps and is highly interpretable. We convert sub reasoning tasks to
acceptable objectives of PTMs and assign tasks to proper PTMs without any
adaptation. Our experiments on two VQA benchmarks under the zero-shot setting
demonstrate the effectiveness of our method and better interpretability
compared with several baselines.

本文探讨如何利用预训练模型来支持零样本视觉问答，通过模块化的零样本网络将问题分解成子理性步骤，并将子任务分配给适当的预训练模型以实现更好的可解释性。实验表明，我们的方法比其他基线方法更具有效性和可解释性。