Generally, the decoder-only large language models (LLMs) are adapted to
context-aware neural machine translation (NMT) in a concatenating way, where
LLMs take the concatenation of the source sentence (i.e., intra-sentence
context) and the inter-sentence context as the input, and then to generate the
target tokens sequentially. This adaptation strategy, i.e., concatenation mode,
considers intra-sentence and inter-sentence contexts with the same priority,
despite an apparent difference between the two kinds of contexts. In this
paper, we propose an alternative adaptation approach, named Decoding-enhanced
Multi-phase Prompt Tuning (DeMPT), to make LLMs discriminately model and
utilize the inter- and intra-sentence context and more effectively adapt LLMs
to context-aware NMT. First, DeMPT divides the context-aware NMT process into
three separate phases. During each phase, different continuous prompts are
introduced to make LLMs discriminately model various information. Second, DeMPT
employs a heuristic way to further discriminately enhance the utilization of
the source-side inter- and intra-sentence information at the final decoding
phase. Experiments show that our approach significantly outperforms the
concatenation method, and further improves the performance of LLMs in discourse
modeling.

该论文通过提出一种名为 Decoding-enhanced Multi-phase Prompt Tuning（DeMPT）的选择性适应方法，使得仅通过解码器的大型语言模型能够有针对性地模拟和利用句内和句间上下文，并在语篇建模中显著改进模型性能。

DeMPT: 解码增强的多阶段提示调整，使 LLM 更好地理解上下文并提供翻译

DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be  Better Context-aware Translators

Large language models (LLMs) such as Chat-GPT can produce coherent, cohesive,
relevant, and fluent answers for various natural language processing (NLP)
tasks. Taking document-level machine translation (MT) as a testbed, this paper
provides an in-depth evaluation of LLMs' ability on discourse modeling. The
study fo-cuses on three aspects: 1) Effects of Discourse-Aware Prompts, where
we investigate the impact of different prompts on document-level translation
quality and discourse phenomena; 2) Comparison of Translation Models, where we
compare the translation performance of Chat-GPT with commercial MT systems and
advanced document-level MT methods; 3) Analysis of Discourse Modelling
Abilities, where we further probe discourse knowledge encoded in LLMs and
examine the impact of training techniques on discourse modeling. By evaluating
a number of benchmarks, we surprisingly find that 1) leveraging their powerful
long-text mod-eling capabilities, ChatGPT outperforms commercial MT systems in
terms of human evaluation. 2) GPT-4 demonstrates a strong ability to explain
discourse knowledge, even through it may select incorrect translation
candidates in contrastive testing. 3) ChatGPT and GPT-4 have demonstrated
superior performance and show potential to become a new and promising paradigm
for document-level translation. This work highlights the challenges and
opportunities of discourse modeling for LLMs, which we hope can inspire the
future design and evaluation of LLMs.

本文研究利用 Chat-GPT 建立的大型语言模型在文档级机器翻译中的应用，通过评估话语建模的能力，比较它与商业翻译系统和高级文档级机器翻译方法的性能，发现 Chat-GPT 在人类评估方面表现优异，同时揭示了话语建模的挑战和机遇。