Situated conversations, which refer to visual information as visual question
answering (VQA), often contain ambiguities caused by reliance on directive
information. This problem is exacerbated because some languages, such as
Japanese, often omit subjective or objective terms. Such ambiguities in
questions are often clarified by the contexts in conversational situations,
such as joint attention with a user or user gaze information. In this study, we
propose the Gaze-grounded VQA dataset (GazeVQA) that clarifies ambiguous
questions using gaze information by focusing on a clarification process
complemented by gaze information. We also propose a method that utilizes gaze
target estimation results to improve the accuracy of GazeVQA tasks. Our
experimental results showed that the proposed method improved the performance
in some cases of a VQA system on GazeVQA and identified some typical problems
of GazeVQA tasks that need to be improved.

通过利用注视信息澄清有歧义的问题，我们提出了以注视为基础的视觉问题回答数据集 (GazeVQA)，并提出了一种利用注视目标估计结果提高 GazeVQA 任务准确性的方法。实验结果显示该方法在某些情况下提高了 VQA 系统在 GazeVQA 上的表现，并识别了需要改进的 GazeVQA 任务的一些典型问题。