The transformer has been shown to outperform recurrent neural network-based
sequence-to-sequence models in various word-level NLP tasks. Yet for
character-level transduction tasks, e.g. morphological inflection generation
and historical text normalization, there are few works that outperform
recurrent models using the transformer. In an empirical study, we uncover that,
in contrast to recurrent sequence-to-sequence models, the batch size plays a
crucial role in the performance of the transformer on character-level tasks,
and we show that with a large enough batch size, the transformer does indeed
outperform recurrent models. We also introduce a simple technique to handle
feature-guided character-level transduction that further improves performance.
With these insights, we achieve state-of-the-art performance on morphological
inflection and historical text normalization. We also show that the transformer
outperforms a strong baseline on two other character-level transduction tasks:
grapheme-to-phoneme conversion and transliteration.

这项研究比较了循环神经网络和 transformer 在字符级转换任务方面的表现，发现 batch size 在 transformer 的性能上扮演了关键的角色，在足够大的 batch size 下，transformer 的表现强于循环模型。此外，还提出了一种简单的方法来处理特征导向的字符级转换，并在形态学屈折和历史文本规范化两个任务上实现了最先进的表现。实验结果表明，与强基准相比，transformer 在两个其他字符级转换任务上的表现有所提高：字素到音素的转换和音译。

将 Transformer 应用于字符级转换

Applying the Transformer to Character-level Transduction

There is no consensus on the state-of-the-art approach to historical text
normalization. Many techniques have been proposed, including rule-based
methods, distance metrics, character-based statistical machine translation, and
neural encoder--decoder models, but studies have used different datasets,
different evaluation methods, and have come to different conclusions. This
paper presents the largest study of historical text normalization done so far.
We critically survey the existing literature and report experiments on eight
languages, comparing systems spanning all categories of proposed normalization
techniques, analysing the effect of training data quantity, and using different
evaluation methods. The datasets and scripts are made publicly available.

本文介绍迄今最大的历史文本规范化研究，涵盖了基于规则、距离度量、基于字符的机器翻译、神经编码器 - 解码器模型等所有提出的规范化技术类别，并使用不同的数据集和评估方法进行比较，分析了训练数据数量的影响，并提供了公开的数据集和脚本。