We propose a framework for sequence-to-sequence contrastive learning (SeqCLR)
of visual representations, which we apply to text recognition. To account for
the sequence-to-sequence structure, each feature map is divided into different
instances over which the contrastive loss is computed. This operation enables
us to contrast in a sub-word level, where from each image we extract several
positive pairs and multiple negative examples. To yield effective visual
representations for text recognition, we further suggest novel augmentation
heuristics, different encoder architectures and custom projection heads.
Experiments on handwritten text and on scene text show that when a text decoder
is trained on the learned representations, our method outperforms
non-sequential contrastive methods. In addition, when the amount of supervision
is reduced, SeqCLR significantly improves performance compared with supervised
training, and when fine-tuned with 100% of the labels, our method achieves
state-of-the-art results on standard handwritten text recognition benchmarks.

提出一种基于序列对序列对比学习框架 (SeqCLR) 的视觉表征方法，应用于文本识别，可实现在子单词级别上对比，并进一步建议新颖的数据增广启发式方法、不同的编码器架构和定制投影头来获得对于文本识别而言的有效视觉表征。在手写文本和场景文本上的实验结果表明，与非序列对比方法相比，当训练文本解码器的表征时，我们的方法表现出更好的性能。另外，当减少监督量时，SeqCLR 与有监督的训练相比，明显提高了性能，在 100％的标签下微调后，我们的方法在标准手写文本识别基准上取得了最佳结果。