While supervised fine-tuning (SFT) has been a straightforward approach for
tailoring the output of foundation large language model (LLM) to specific
preferences, concerns have been raised about the depth of this alignment, with
some critiques suggesting it is merely "superficial". We critically examine
this hypothesis within the scope of cross-lingual generation tasks, proposing
that the effectiveness of SFT may be constrained by its reliance on prior
tokens to guide cross-lingual generation. Based on this crucial insight, and in
response to the challenges posed by the costly and limited availability of
non-English data for SFT, we introduce a novel training-free alignment method
named PreTTY, which employs minimal task-related prior tokens to bridge the
foundation LLM and the SFT LLM, achieving comparable performance without
training. Experiments on machine translation and part-of-speech tagging across
eight languages demonstrate the efficacy of PreTTY in cross-lingual settings.
Remarkably, by initiating the decoding process with only one or two prior
tokens, foundation LLMs can achieve performance comparable to their SFT
counterparts. This method presents a cost-effective alternative to SFT and
advances the democratization of multilingual LLMs.

基于对有关预先令牌指导跨语言生成的依赖性的重要洞察，我们提出了一种名为 PreTTY 的新型无需训练的对齐方法，用于在基础 LLM 和 SFT LLM 之间建立桥梁，以实现可比较的性能。在八种语言上进行的机器翻译和词性标注实验表明，PreTTY 在跨语言环境中具有很高的效能。这种方法是 SFT 的一种经济高效的替代方案，推动了多语言 LLM 的民主化。