Visual grounding is a task to locate the target indicated by a natural
language expression. Existing methods extend the generic object detection
framework to this problem. They base the visual grounding on the features from
pre-generated proposals or anchors, and fuse these features with the text
embeddings to locate the target mentioned by the text. However, modeling the
visual features from these predefined locations may fail to fully exploit the
visual context and attribute information in the text query, which limits their
performance. In this paper, we propose a transformer-based framework for
accurate visual grounding by establishing text-conditioned discriminative
features and performing multi-stage cross-modal reasoning. Specifically, we
develop a visual-linguistic verification module to focus the visual features on
regions relevant to the textual descriptions while suppressing the unrelated
areas. A language-guided feature encoder is also devised to aggregate the
visual contexts of the target object to improve the object's distinctiveness.
To retrieve the target from the encoded visual features, we further propose a
multi-stage cross-modal decoder to iteratively speculate on the correlations
between the image and text for accurate target localization. Extensive
experiments on five widely used datasets validate the efficacy of our proposed
components and demonstrate state-of-the-art performance. Our code is public at
this https URL

本研究提出了一种基于 transformer 的视觉定位框架，通过建立文本条件的区分性特征和执行多阶段跨模态推理来实现精确的视觉定位，并提出了基于文本的视觉上下文信息编码器和多阶段解码器以实现最新的性能。