Modern large language models use a fixed tokenizer to effectively compress text drawn from a source domain. However, applying the same tokenizer to a new target domain often leads to inferior compression, more costly inference, and reduced semantic alignment. To address this deficiency, we introduce Sparse Sinkhorn Token Translation (S2T2). S2T2 trains a tailored tokenizer for the target domain and learns to translate between target and source tokens, enabling more effective reuse of the pre-trained next-source-token predictor. In our experiments with finetuned English language models, S2T2 improves both the perplexity and the compression of out-of-domain protein sequences, outperforming direct finetuning with either the source or target tokenizer. In addition, we find that token translations learned for smaller, less expensive models can be directly transferred to larger, more powerful models to reap the benefits of S2T2 at lower cost.

该研究解决了在新的目标领域中应用固定标记器导致的文本压缩效果差和推理成本高的问题。提出的稀疏Sinkhorn标记翻译（S2T2）方法，通过为目标领域训练量身定制的标记器，实现了源标记与目标标记之间的有效转换，显著提高了域外蛋白质序列的困惑度和压缩效果，且学习到的标记翻译可以有效迁移到更大的模型，降低成本。 

通过标记翻译调整语言模型