The relationship between language model tokenization and performance is an
open area of research. Here, we investigate how different tokenization schemes
impact number agreement in Spanish plurals. We find that
morphologically-aligned tokenization performs similarly to other tokenization
schemes, even when induced artificially for words that would not be tokenized
that way during training. We then present exploratory analyses demonstrating
that language model embeddings for different plural tokenizations have similar
distributions along the embedding space axis that maximally distinguishes
singular and plural nouns. Our results suggest that morphologically-aligned
tokenization is a viable tokenization approach, and existing models already
generalize some morphological patterns to new items. However, our results
indicate that morphological tokenization is not strictly required for
performance.

语言模型标记化与性能之间的关系是一个开放的研究领域。我们研究了不同标记化方案如何影响西班牙语复数的数一致性。我们发现，形态对齐标记化与其他标记化方案的表现相似，即使在对训练过程中不会进行这种标记化的单词人工引导下。我们还进行了探索性分析，显示了不同复数标记化的语言模型嵌入在嵌入空间中具有类似的分布，最大化区分单数名词和复数名词的嵌入轴线。我们的结果表明，形态对齐标记化是一种可行的标记化方法，现有模型已经对一些形态模式进行了泛化。然而，我们的结果表明形态标记化并不是性能的严格要求。