Language tasks involving character-level manipulations (e.g., spelling
correction, many word games) are challenging for models based in subword
tokenization. To address this, we adapt the interchange intervention training
method of Geiger et al. (2021) to operate on type-level variables over
characters. This allows us to encode robust, position-independent
character-level information in the internal representations of subword-based
models. We additionally introduce a suite of character-level tasks that
systematically vary in their dependence on meaning and sequence-level context.
While simple character-level tokenization approaches still perform best on
purely form-based tasks like string reversal, our method is superior for more
complex tasks that blend form, meaning, and context, such as spelling
correction in context and word search games. Our approach also leads to
subword-based models with human-intepretable internal representations of
characters.

该论文介绍了基于字符级别的操作（如拼写纠正、字谜游戏）对于基于子单词分词的模型非常具有挑战性。为了解决这个问题，作者采用了 Geiger 等人 2021 年的交互式干预训练方法，将其适应到基于字符的类型变量上。该方法可以在子单词分词模型的内部表示中编码稳健的、位置独立的字符级别信息。作者还介绍了一套字符级别的任务，这些任务在依赖于含义和序列级别上下文的程度上有所不同。虽然对于纯粹的形式任务（如字符串翻转）来说，简单的字符级别分词方法仍然表现最佳，但我们的方法对于更复杂的任务，如上下文中的拼写纠正和文字搜索游戏，则更加优越。我们的方法还导致了基于子单词的模型具有人类可解释的内部表示形式。