Although text recognition has significantly evolved over the years,
state-of-the-art (SOTA) models still struggle in the wild scenarios due to
complex backgrounds, varying fonts, uncontrolled illuminations, distortions and
other artefacts. This is because such models solely depend on visual
information for text recognition, thus lacking semantic reasoning capabilities.
In this paper, we argue that semantic information offers a complementary role
in addition to visual only. More specifically, we additionally utilize semantic
information by proposing a multi-stage multi-scale attentional decoder that
performs joint visual-semantic reasoning. Our novelty lies in the intuition
that for text recognition, the prediction should be refined in a stage-wise
manner. Therefore our key contribution is in designing a stage-wise unrolling
attentional decoder where non-differentiability, invoked by discretely
predicted character labels, needs to be bypassed for end-to-end training. While
the first stage predicts using visual features, subsequent stages refine on top
of it using joint visual-semantic information. Additionally, we introduce
multi-scale 2D attention along with dense and residual connections between
different stages to deal with varying scales of character sizes, for better
performance and faster convergence during training. Experimental results show
our approach to outperform existing SOTA methods by a considerable margin.

通过引入语义信息，该论文提出了一种基于多级多尺度注意力解码器的视觉 - 语义推理方法，使用以往的视觉特征进行预测并使用多级注意力进行精细化的预测，获得了比现有技术达到了更好的性能。