Recent works have shown huge success of deep learning models for common in
vocabulary (IV) scene text recognition. However, in real-world scenarios,
out-of-vocabulary (OOV) words are of great importance and SOTA recognition
models usually perform poorly on OOV settings. Inspired by the intuition that
the learned language prior have limited OOV preformence, we design a framework
named Vision Language Adaptive Mutual Decoder (VLAMD) to tackle OOV problems
partly. VLAMD consists of three main conponents. Firstly, we build an attention
based LSTM decoder with two adaptively merged visual-only modules, yields a
vision-language balanced main branch. Secondly, we add an auxiliary query based
autoregressive transformer decoding head for common visual and language prior
representation learning. Finally, we couple these two designs with
bidirectional training for more diverse language modeling, and do mutual
sequential decoding to get robuster results. Our approach achieved 70.31\% and
59.61\% word accuracy on IV+OOV and OOV settings respectively on Cropped Word
Recognition Task of OOV-ST Challenge at ECCV 2022 TiE Workshop, where we got
1st place on both settings.

该研究提出了一种名为 VLAMD 的框架，该框架基于深度学习，利用语言先验知识，设计了一种针对 OCR 中存在的 Out-of-vocabulary 问题的解决方案，其中包括注意力 LSTM 解码器，自回归转换器解码器和双向训练方法，该方法在 2022 ECCV TiE Workshop 的 OOV-ST 挑战中表现出良好的性能。