Text-based visual question answering (TextVQA) faces the significant
challenge of avoiding redundant relational inference. To be specific, a large
number of detected objects and optical character recognition (OCR) tokens
result in rich visual relationships. Existing works take all visual
relationships into account for answer prediction. However, there are three
observations: (1) a single subject in the images can be easily detected as
multiple objects with distinct bounding boxes (considered repetitive objects).
The associations between these repetitive objects are superfluous for answer
reasoning; (2) two spatially distant OCR tokens detected in the image
frequently have weak semantic dependencies for answer reasoning; and (3) the
co-existence of nearby objects and tokens may be indicative of important visual
cues for predicting answers. Rather than utilizing all of them for answer
prediction, we make an effort to identify the most important connections or
eliminate redundant ones. We propose a sparse spatial graph network (SSGN) that
introduces a spatially aware relation pruning technique to this task. As
spatial factors for relation measurement, we employ spatial distance, geometric
dimension, overlap area, and DIoU for spatially aware pruning. We consider
three visual relationships for graph learning: object-object, OCR-OCR tokens,
and object-OCR token relationships. SSGN is a progressive graph learning
architecture that verifies the pivotal relations in the correlated object-token
sparse graph, and then in the respective object-based sparse graph and
token-based sparse graph. Experiment results on TextVQA and ST-VQA datasets
demonstrate that SSGN achieves promising performances. And some visualization
results further demonstrate the interpretability of our method.

TextVQA 中解决冗余关系推理的稀疏空间图网络 (SSGN) 被提出，通过空间感知关系修剪技术识别最重要的连接或消除冗余连接，在图学习中考虑物体 - 物体关系和 OCR-OCR 标记关系。实验证明 SSGN 在 TextVQA 和 ST-VQA 数据集上取得了令人满意的性能表现，并且某些可视化结果进一步证明了我们方法的可解释性。