Tokenization is a crucial step that bridges human-readable text with model-readable discrete tokens. However, recent studies have revealed that tokenizers can be exploited to elicit unwanted model behaviors. In this work, we investigate incomplete tokens, i.e., undecodable tokens with stray bytes resulting from byte-level byte-pair encoding (BPE) tokenization. We hypothesize that such tokens are heavily reliant on their adjacent tokens and are fragile when paired with unfamiliar tokens. To demonstrate this vulnerability, we introduce improbable bigrams: out-of-distribution combinations of incomplete tokens designed to exploit their dependency. Our experiments show that improbable bigrams are significantly prone to hallucinatory behaviors. Surprisingly, alternative tokenizations of the same phrases result in drastically lower rates of hallucination (93% reduction in Llama3.1). We caution against the potential vulnerabilities introduced by byte-level BPE tokenizers, which may impede the development of trustworthy language models.

本研究针对字节级字对编码分词中的不完整标记问题，提出不太可能的双字组概念，以揭示这些标记在与不熟悉的标记配对时的脆弱性。实验表明，这些双字组显著增加了生成虚假内容的风险，而采用其他分词方法则显著降低了这种风险。因此，研究指出字节级BPE分词器可能带来的潜在脆弱性，影响可信语言模型的开发。

不太可能的双字组揭示了字节级分词器中不完整标记的脆弱性