Recent work has validated the importance of subword information for word
representation learning. Since subwords increase parameter sharing ability in
neural models, their value should be even more pronounced in low-data regimes.
In this work, we therefore provide a comprehensive analysis focused on the
usefulness of subwords for word representation learning in truly low-resource
scenarios and for three representative morphological tasks: fine-grained entity
typing, morphological tagging, and named entity recognition. We conduct a
systematic study that spans several dimensions of comparison: 1) type of data
scarcity which can stem from the lack of task-specific training data, or even
from the lack of unannotated data required to train word embeddings, or both;
2) language type by working with a sample of 16 typologically diverse languages
including some truly low-resource ones (e.g. Rusyn, Buryat, and Zulu); 3) the
choice of the subword-informed word representation method. Our main results
show that subword-informed models are universally useful across all language
types, with large gains over subword-agnostic embeddings. They also suggest
that the effective use of subwords largely depends on the language (type) and
the task at hand, as well as on the amount of available data for training the
embeddings and task-based models, where having sufficient in-task data is a
more critical requirement.

本研究探讨了在低资源环境下，子词信息对单词表示学习的有用性以及其在细粒度实体类型、形态标注和命名实体识别等三个典型形态学任务中的应用价值，结果表明，子词信息可以普适地提升单词表示，但其效果受语言类型和任务的影响，同时，训练单词嵌入和基于任务的模型所需的数据越充足，子词信息的效果才会更好。

对于真正低资源语言中形态学任务的子词信息的重要性

On the Importance of Subword Information for Morphological Tasks in  Truly Low-Resource Languages

While machine translation has traditionally relied on large amounts of
parallel corpora, a recent research line has managed to train both Neural
Machine Translation (NMT) and Statistical Machine Translation (SMT) systems
using monolingual corpora only. In this paper, we identify and address several
deficiencies of existing unsupervised SMT approaches by exploiting subword
information, developing a theoretically well founded unsupervised tuning
method, and incorporating a joint refinement procedure. Moreover, we use our
improved SMT system to initialize a dual NMT model, which is further fine-tuned
through on-the-fly back-translation. Together, we obtain large improvements
over the previous state-of-the-art in unsupervised machine translation. For
instance, we get 22.5 BLEU points in English-to-German WMT 2014, 5.5 points
more than the previous best unsupervised system, and 0.5 points more than the
(supervised) shared task winner back in 2014.

本文针对现有无监督机器翻译存在的不足之处，通过利用子词信息、开发理论上有根据的无监督调优方法和引入联合优化程序等手段，改进了统计机器翻译系统，并将其用于初始化双 NMT 模型并进行后续微调，使无监督机器翻译的表现大幅度提升。在英德无监督机器翻译领域，改进后的 SMT 系统相较之前最好的无监督系统 BLEU 值提高了 5.5 个百分点，超过了当时的（监督）任务获胜者。