Data-driven subword segmentation has become the default strategy for
open-vocabulary machine translation and other NLP tasks, but may not be
sufficiently generic for optimal learning of non-concatenative morphology. We
design a test suite to evaluate segmentation strategies on different types of
morphological phenomena in a controlled, semi-synthetic setting. In our
experiments, we compare how well machine translation models trained on subword-
and character-level can translate these morphological phenomena. We find that
learning to analyse and generate morphologically complex surface
representations is still challenging, especially for non-concatenative
morphological phenomena like reduplication or vowel harmony and for rare word
stems. Based on our results, we recommend that novel text representation
strategies be tested on a range of typologically diverse languages to minimise
the risk of adopting a strategy that inadvertently disadvantages certain
languages.

本论文研究基于数据驱动的子词分割在自然语言处理等领域的应用是否适用于非连接语素构词方式，通过一套测试套件对分割策略在不同形态学现象上的表现进行比较，并发现学习分析和生成表面的形态上下文仍然具有挑战性且应针对多种语言应用测试新的文本表征策略，以减少不同策略对某些语言造成的不利影响。