Knowledge-based visual question answering (KVQA) task aims to answer
questions that require additional external knowledge as well as an
understanding of images and questions. Recent studies on KVQA inject an
external knowledge in a multi-modal form, and as more knowledge is used,
irrelevant information may be added and can confuse the question answering. In
order to properly use the knowledge, this study proposes the following: 1) we
introduce a novel semantic inconsistency measure computed from caption
uncertainty and semantic similarity; 2) we suggest a new external knowledge
assimilation method based on the semantic inconsistency measure and apply it to
integrate explicit knowledge and implicit knowledge for KVQA; 3) the proposed
method is evaluated with the OK-VQA dataset and achieves the state-of-the-art
performance.

本研究提出了一种基于语义不一致度量的新型外部知识同化方法，应用到知识型视觉问答中，旨在减少添加无关信息对问答的干扰并融合含蓄和显式知识，评估结果表明该方法具有最先进性能。