Neural sequence-to-sequence models are currently the predominant choice for
language generation tasks. Yet, on word-level tasks, exact inference of these
models reveals the empty string is often the global optimum. Prior works have
speculated this phenomenon is a result of the inadequacy of neural models for
language generation. However, in the case of morphological inflection, we find
that the empty string is almost never the most probable solution under the
model. Further, greedy search often finds the global optimum. These
observations suggest that the poor calibration of many neural models may stem
from characteristics of a specific subset of tasks rather than general
ill-suitedness of such models for language generation.

该研究探讨了神经序列到序列模型在语言生成任务中的应用，发现在单词级任务中，准确推断这些模型会发现空字符串常常是全局最优解，但在形态学变化的情况下，空字符串几乎不是模型最可能的解，且贪婪搜索通常能找到全局最优解。这表明，许多神经模型的不良校准可能来自于特定任务的特征，而非这些模型在语言生成方面的普遍不适用性。