Soft Prompt Tuning (SPT) is a parameter-efficient method for adapting pre-trained language models (PLMs) to specific tasks by inserting learnable embeddings, or soft prompts, at the input layer of the PLM, without modifying its parameters. This paper investigates the potential of SPT for cross-lingual transfer. Unlike previous studies on SPT for cross-lingual transfer that often fine-tune both the soft prompt and the model parameters, we adhere to the original intent of SPT by keeping the model parameters frozen and only training the soft prompt. This does not only reduce the computational cost and storage overhead of full-model fine-tuning, but we also demonstrate that this very parameter efficiency intrinsic to SPT can enhance cross-lingual transfer performance to linguistically distant languages. Moreover, we explore how different factors related to the prompt, such as the length or its reparameterization, affect cross-lingual transfer performance.

通过插入可学习的嵌入或软提示到预训练语言模型 (PLM) 的输入层，Soft Prompt Tuning (SPT) 是一种将 PLM 调适到特定任务的参数高效方法，无需修改其参数。本文研究了 SPT 在跨语言传递中的潜力，并通过冻结模型参数并只训练软提示以保持 SPT 的参数高效性，不仅减少了计算成本和存储开销，还证明了这一特性能够增强对语言上远离的语言的跨语言传递性能。此外，我们还探索了与软提示相关的不同因素（如长度或重新参数化）对跨语言传递性能的影响。

跨语言迁移的软提示调整：少即是多