Text spotting end-to-end methods have recently gained attention in the
literature due to the benefits of jointly optimizing the text detection and
recognition components. Existing methods usually have a distinct separation
between the detection and recognition branches, requiring exact annotations for
the two tasks. We introduce TextTranSpotter (TTS), a transformer-based approach
for text spotting and the first text spotting framework which may be trained
with both fully- and weakly-supervised settings. By learning a single latent
representation per word detection, and using a novel loss function based on the
Hungarian loss, our method alleviates the need for expensive localization
annotations. Trained with only text transcription annotations on real data, our
weakly-supervised method achieves competitive performance with previous
state-of-the-art fully-supervised methods. When trained in a fully-supervised
manner, TextTranSpotter shows state-of-the-art results on multiple benchmarks.

介绍一种基于 transformer 的文本定位方法 TextTranSpotter (TTS)，该方法可以在全监督和弱监督的情况下训练，并使用基于 Hungarian loss 的新型损失函数，不需要昂贵的本地化注释，通过每个单词检测学习单一潜在表示，实现了与先前最先进的全监督方法相当的性能。

使用多任务变压器实现弱监督文本识别

Towards Weakly-Supervised Text Spotting using a Multi-Task Transformer

DETR is a recently proposed Transformer-based method which views object
detection as a set prediction problem and achieves state-of-the-art performance
but demands extra-long training time to converge. In this paper, we investigate
the causes of the optimization difficulty in the training of DETR. Our
examinations reveal several factors contributing to the slow convergence of
DETR, primarily the issues with the Hungarian loss and the Transformer
cross-attention mechanism. To overcome these issues we propose two solutions,
namely, TSP-FCOS (Transformer-based Set Prediction with FCOS) and TSP-RCNN
(Transformer-based Set Prediction with RCNN). Experimental results show that
the proposed methods not only converge much faster than the original DETR, but
also significantly outperform DETR and other baselines in terms of detection
accuracy.

本文讨论了 DETR 在优化过程中所面对的问题，提出了 TSP-FCOS 和 TSP-RCNN 这两种方法以解决其中与匈牙利损失和 Transformer 跨注意机制有关的问题，并证实了这两个新方法在收敛速度和检测准确度上都显著优于 DETR 和其他基准模型。