Recently, vision-language joint representation learning has proven to be
highly effective in various scenarios. In this paper, we specifically adapt
vision-language joint learning for scene text detection, a task that
intrinsically involves cross-modal interaction between the two modalities:
vision and language, since text is the written form of language. Concretely, we
propose to learn contextualized, joint representations through vision-language
pre-training, for the sake of enhancing the performance of scene text
detectors. Towards this end, we devise a pre-training architecture with an
image encoder, a text encoder and a cross-modal encoder, as well as three
pretext tasks: image-text contrastive learning (ITC), masked language modeling
(MLM) and word-in-image prediction (WIP). The pre-trained model is able to
produce more informative representations with richer semantics, which could
readily benefit existing scene text detectors (such as EAST and PSENet) in the
down-stream text detection task. Extensive experiments on standard benchmarks
demonstrate that the proposed paradigm can significantly improve the
performance of various representative text detectors, outperforming previous
pre-training approaches. The code and pre-trained models will be publicly
released.

本文研究了视觉 - 语言联合表征学习在场景文本检测中的应用，提出了利用视觉 - 语言预训练学习得到的上下文化联合表征来提高场景文本检测器的性能，通过三个 pretext tasks 来进行预训练，在标准基准测试中证明了该方法能够显著提高各种代表性的文本检测器的性能。