Offline handwriting recognition (HWR) has improved significantly with the
advent of deep learning architectures in recent years. Nevertheless, it remains
a challenging problem and practical applications often rely on post-processing
techniques for restricting the predicted words via lexicons or language models.
Despite their enhanced performance, such systems are less usable in contexts
where out-of-vocabulary words are anticipated, e.g. for detecting misspelled
words in school assessments. To that end, we introduce the task of comparing a
handwriting image to text. To solve the problem, we propose an unrestricted
binary classifier, consisting of a HWR feature extractor and a multimodal
classification head which convolves the feature extractor output with the
vector representation of the input text. Our model's classification head is
trained entirely on synthetic data created using a state-of-the-art generative
adversarial network. We demonstrate that, while maintaining high recall, the
classifier can be calibrated to achieve an average precision increase of 19.5%
compared to addressing the task by directly using state-of-the-art HWR models.
Such massive performance gains can lead to significant productivity increases
in applications utilizing human-in-the-loop automation.

通过引入与文本进行比较的任务，我们提出了一个无限制的二元分类器，其中包括一个手写识别特征提取器和一个多模态分类头，该分类头将特征提取器的输出与输入文本的向量表示进行卷积。我们的模型的分类头完全基于使用最先进的生成对抗网络创建的合成数据进行训练。我们证明，与直接使用最先进的手写识别模型来解决任务相比，尽管保持高回收率，但分类器可以进行校准，使平均精确度提高了 19.5%。这种巨大的性能提升可以在利用人 - 环自动化的应用中带来显著的生产力增长。

卷积识别模型特征与文本标签相结合的离线检测错误拼写的手写词

Offline Detection of Misspelled Handwritten Words by Convolving  Recognition Model Features with Text Labels

Offline handwriting recognition systems require cropped text line images for
both training and recognition. On the one hand, the annotation of position and
transcript at line level is costly to obtain. On the other hand, automatic line
segmentation algorithms are prone to errors, compromising the subsequent
recognition. In this paper, we propose a modification of the popular and
efficient multi-dimensional long short-term memory recurrent neural networks
(MDLSTM-RNNs) to enable end-to-end processing of handwritten paragraphs. More
particularly, we replace the collapse layer transforming the two-dimensional
representation into a sequence of predictions by a recurrent version which can
recognize one line at a time. In the proposed model, a neural network performs
a kind of implicit line segmentation by computing attention weights on the
image representation. The experiments on paragraphs of Rimes and IAM database
yield results that are competitive with those of networks trained at line
level, and constitute a significant step towards end-to-end transcription of
full documents.

本文提出了一种针对离线手写段落文本识别的神经网络模型，利用注意力权重计算图像表示，实现了一种隐式行分割的可训练端到端模型，试验结果表明其性能与传统基于线段的模型竞争力相当，具有将全文档转录实现的潜力。