Visual question answering is a multimodal task that requires the joint
comprehension of visual and textual information. However, integrating visual
and textual semantics solely through attention layers is insufficient to
comprehensively understand and align information from both modalities.
Intuitively, object attributes can naturally serve as a bridge to unify them,
which has been overlooked in previous research. In this paper, we propose a
novel VQA approach from the perspective of utilizing object attribute, aiming
to achieve better object-level visual-language alignment and multimodal scene
understanding. Specifically, we design an attribute fusion module and a
contrastive knowledge distillation module. The attribute fusion module
constructs a multimodal graph neural network to fuse attributes and visual
features through message passing. The enhanced object-level visual features
contribute to solving fine-grained problem like counting-question. The better
object-level visual-language alignment aids in understanding multimodal scenes,
thereby improving the model's robustness. Furthermore, to augment scene
understanding and the out-of-distribution performance, the contrastive
knowledge distillation module introduces a series of implicit knowledge. We
distill knowledge into attributes through contrastive loss, which further
strengthens the representation learning of attribute features and facilitates
visual-linguistic alignment. Intensive experiments on six datasets, COCO-QA,
VQAv2, VQA-CPv2, VQA-CPv1, VQAvs and TDIUC, show the superiority of the
proposed method.

提出了一种基于物体属性利用的视觉问答方法，旨在实现更好的物体级视觉 - 语言对齐和多模态场景理解。通过属性融合模块和对比知识蒸馏模块，构建多模态图神经网络，融合属性和视觉特征，改善细粒度问题的解决和多模态场景的理解，从而提高模型的鲁棒性。通过对六个数据集的密集实验，COCO-QA、VQAv2、VQA-CPv2、VQA-CPv1、VQAvs 和 TDIUC，验证了该方法的优越性。