Text-based Visual Question Answering (TextVQA) aims at answering questions
about the text in images. Most works in this field focus on designing network
structures or pre-training tasks. All these methods list the OCR texts in
reading order (from left to right and top to bottom) to form a sequence, which
is treated as a natural language ``sentence''. However, they ignore the fact
that most OCR words in the TextVQA task do not have a semantical contextual
relationship. In addition, these approaches use 1-D position embedding to
construct the spatial relation between OCR tokens sequentially, which is not
reasonable. The 1-D position embedding can only represent the left-right
sequence relationship between words in a sentence, but not the complex spatial
position relationship. To tackle these problems, we propose a novel method
named Separate and Locate (SaL) that explores text contextual cues and designs
spatial position embedding to construct spatial relations between OCR texts.
Specifically, we propose a Text Semantic Separate (TSS) module that helps the
model recognize whether words have semantic contextual relations. Then, we
introduce a Spatial Circle Position (SCP) module that helps the model better
construct and reason the spatial position relationships between OCR texts. Our
SaL model outperforms the baseline model by 4.44% and 3.96% accuracy on TextVQA
and ST-VQA datasets. Compared with the pre-training state-of-the-art method
pre-trained on 64 million pre-training samples, our method, without any
pre-training tasks, still achieves 2.68% and 2.52% accuracy improvement on
TextVQA and ST-VQA. Our code and models will be released at
this https URL

TextVQA 问题解答中，我们提出了一种名为 Separate and Locate (SaL) 的新方法，该方法通过探索文本的上下文线索和设计空间位置嵌入来构建 OCR 文本之间的空间关系，从而在 TextVQA 和 ST-VQA 数据集上获得了较好的准确性改进，并且无需任何预训练任务。