Answer grounding is the task of locating relevant visual evidence for the
Visual Question Answering task. While a wide variety of attention methods have
been introduced for this task, they suffer from the following three problems:
designs that do not allow the usage of pre-trained networks and do not benefit
from large data pre-training, custom designs that are not based on
well-grounded previous designs, therefore limiting the learning power of the
network, or complicated designs that make it challenging to re-implement or
improve them. In this paper, we propose a novel architectural block, which we
term Sentence Attention Block, to solve these problems. The proposed block
re-calibrates channel-wise image feature-maps by explicitly modeling
inter-dependencies between the image feature-maps and sentence embedding. We
visually demonstrate how this block filters out irrelevant feature-maps
channels based on sentence embedding. We start our design with a well-known
attention method, and by making minor modifications, we improve the results to
achieve state-of-the-art accuracy. The flexibility of our method makes it easy
to use different pre-trained backbone networks, and its simplicity makes it
easy to understand and be re-implemented. We demonstrate the effectiveness of
our method on the TextVQA-X, VQS, VQA-X, and VizWiz-VQA-Grounding datasets. We
perform multiple ablation studies to show the effectiveness of our design
choices.

提出了一种名为 “Sentence Attention Block” 的新型建筑模块，通过明确建模图像特征图与句子嵌入之间的相互依赖关系，重新校准了通道级图像特征图，以过滤基于句子嵌入的不相关特征图通道，进而实现了目前最先进准确度的提高。

用于答案定位的句子注意力模块

Sentence Attention Blocks for Answer Grounding

Answer grounding aims to reveal the visual evidence for visual question
answering (VQA), which entails highlighting relevant positions in the image
when answering questions about images. Previous attempts typically tackle this
problem using pretrained object detectors, but without the flexibility for
objects not in the predefined vocabulary. However, these black-box methods
solely concentrate on the linguistic generation, ignoring the visual
interpretability. In this paper, we propose Dual Visual-Linguistic Interaction
(DaVI), a novel unified end-to-end framework with the capability for both
linguistic answering and visual grounding. DaVI innovatively introduces two
visual-linguistic interaction mechanisms: 1) visual-based linguistic encoder
that understands questions incorporated with visual features and produces
linguistic-oriented evidence for further answer decoding, and 2)
linguistic-based visual decoder that focuses visual features on the
evidence-related regions for answer grounding. This way, our approach ranked
the 1st place in the answer grounding track of 2022 VizWiz Grand Challenge.

本文提出了一种名为 DaVI 的新型统一端到端框架，具有语言回答和视觉定位的能力，引入了两种视觉 - 语言交互机制来解决答案定位问题，并在 2022 年 VizWiz 大挑战中获得了第一名。