Large language models (LLMs) have achieved state-of-the-art performance in machine translation (MT) and demonstrated the ability to leverage in-context learning through few-shot examples. However, the mechanisms by which LLMs use different parts of the input context remain largely unexplored. In this work, we provide a comprehensive analysis of context utilization in MT, studying how LLMs use various context parts, such as few-shot examples and the source text, when generating translations. We highlight several key findings: (1) the source part of few-shot examples appears to contribute more than its corresponding targets, irrespective of translation direction; (2) finetuning LLMs with parallel data alters the contribution patterns of different context parts; and (3) there is a positional bias where earlier few-shot examples have higher contributions to the translated sequence. Finally, we demonstrate that inspecting anomalous context contributions can potentially uncover pathological translations, such as hallucinations. Our findings shed light on the internal workings of LLM-based MT which go beyond those known for standard encoder-decoder MT models.

本研究解决了大型语言模型在机器翻译中如何利用输入上下文的机制未被充分探讨的问题。通过全面分析上下文的利用，研究发现源文本在少量示例中的贡献大于目标文本，微调模型会改变上下文的贡献模式，且早期的少量示例对翻译序列贡献更大。这些发现有助于深入理解基于大语言模型的机器翻译内部机制，并可能揭示翻译中的异常现象。

基于大语言模型的机器翻译中的上下文贡献分析