Pre-trained language-vision models have shown remarkable performance on the
visual question answering (VQA) task. However, most pre-trained models are
trained by only considering monolingual learning, especially the resource-rich
language like English. Training such models for multilingual setups demand high
computing resources and multilingual language-vision dataset which hinders
their application in practice. To alleviate these challenges, we propose a
knowledge distillation approach to extend an English language-vision model
(teacher) into an equally effective multilingual and code-mixed model
(student). Unlike the existing knowledge distillation methods, which only use
the output from the last layer of the teacher network for distillation, our
student model learns and imitates the teacher from multiple intermediate layers
(language and vision encoders) with appropriately designed distillation
objectives for incremental knowledge extraction. We also create the large-scale
multilingual and code-mixed VQA dataset in eleven different language setups
considering the multiple Indian and European languages. Experimental results
and in-depth analysis show the effectiveness of the proposed VQA model over the
pre-trained language-vision models on eleven diverse language setups.

提出一种基于知识蒸馏方法的英语语言 - 视觉模型拓展到同等效果的多语种和混合编码模型的方法，其使用多个中间层来逐步提取知识，同时在 11 种不同语言设置下创建大规模的多语种和混合编码的 VQA 数据集。