A novel scene text recognizer based on Vision-Language Transformer (VLT) is
presented. Inspired by Levenshtein Transformer in the area of NLP, the proposed
method (named Levenshtein OCR, and LevOCR for short) explores an alternative
way for automatically transcribing textual content from cropped natural images.
Specifically, we cast the problem of scene text recognition as an iterative
sequence refinement process. The initial prediction sequence produced by a pure
vision model is encoded and fed into a cross-modal transformer to interact and
fuse with the visual features, to progressively approximate the ground truth.
The refinement process is accomplished via two basic character-level
operations: deletion and insertion, which are learned with imitation learning
and allow for parallel decoding, dynamic length change and good
interpretability. The quantitative experiments clearly demonstrate that LevOCR
achieves state-of-the-art performances on standard benchmarks and the
qualitative analyses verify the effectiveness and advantage of the proposed
LevOCR algorithm. Code is available at
this https URL

提出了一种基于 Vision-Language Transformer (VLT) 的新型场景文本识别器 (Levenshtein OCR)，利用交叉模态 Transformer 相互融合和逐步近似 ground truth, 通过模拟学习，实现并行解码，动态长度变化和良好的可解释性。实验证明，LevOCR 在标准基准测试中实现了最新水平，LevOCR 算法的有效性和优势得到了定量和定性分析的验证。