The use of subword-level information (e.g., characters, character n-grams,
morphemes) has become ubiquitous in modern word representation learning. Its
importance is attested especially for morphologically rich languages which
generate a large number of rare words. Despite a steadily increasing interest
in such subword-informed word representations, their systematic comparative
analysis across typologically diverse languages and different tasks is still
missing. In this work, we deliver such a study focusing on the variation of two
crucial components required for subword-level integration into word
representation models: 1) segmentation of words into subword units, and 2)
subword composition functions to obtain final word representations. We propose
a general framework for learning subword-informed word representations that
allows for easy experimentation with different segmentation and composition
components, also including more advanced techniques based on position
embeddings and self-attention. Using the unified framework, we run experiments
over a large number of subword-informed word representation configurations (60
in total) on 3 tasks (general and rare word similarity, dependency parsing,
fine-grained entity typing) for 5 languages representing 3 language types. Our
main results clearly indicate that there is no "one-sizefits-all"
configuration, as performance is both language- and task-dependent. We also
show that configurations based on unsupervised segmentation (e.g., BPE,
Morfessor) are sometimes comparable to or even outperform the ones based on
supervised word segmentation.

本研究通过对五种语言的三个任务进行大量实验研究，发现在利用子词级别信息进行词表示学习时，不同语言和任务下，分词和组成函数等关键组件的最优配置各异。同时，我们还发现，一些无监督分词方法，如 BPE 和 Morfessor 等，有时能够和甚至超过一些基于监督分词方法的配置。