Generally, the decoder-only large language models (LLMs) are adapted to
context-aware neural machine translation (NMT) in a concatenating way, where
LLMs take the concatenation of the source sentence (i.e., intra-sentence
context) and the inter-sentence context as the input, and then to generate the
target tokens sequentially. This adaptation strategy, i.e., concatenation mode,
considers intra-sentence and inter-sentence contexts with the same priority,
despite an apparent difference between the two kinds of contexts. In this
paper, we propose an alternative adaptation approach, named Decoding-enhanced
Multi-phase Prompt Tuning (DeMPT), to make LLMs discriminately model and
utilize the inter- and intra-sentence context and more effectively adapt LLMs
to context-aware NMT. First, DeMPT divides the context-aware NMT process into
three separate phases. During each phase, different continuous prompts are
introduced to make LLMs discriminately model various information. Second, DeMPT
employs a heuristic way to further discriminately enhance the utilization of
the source-side inter- and intra-sentence information at the final decoding
phase. Experiments show that our approach significantly outperforms the
concatenation method, and further improves the performance of LLMs in discourse
modeling.

该论文通过提出一种名为 Decoding-enhanced Multi-phase Prompt Tuning（DeMPT）的选择性适应方法，使得仅通过解码器的大型语言模型能够有针对性地模拟和利用句内和句间上下文，并在语篇建模中显著改进模型性能。

DeMPT: 解码增强的多阶段提示调整，使 LLM 更好地理解上下文并提供翻译

DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be  Better Context-aware Translators

Standard context-aware neural machine translation (NMT) typically relies on
parallel document-level data, exploiting both source and target contexts.
Concatenation-based approaches in particular, still a strong baseline for
document-level NMT, prepend source and/or target context sentences to the
sentences to be translated, with model variants that exploit equal amounts of
source and target data on each side achieving state-of-the-art results. In this
work, we investigate whether target data should be further promoted within
standard concatenation-based approaches, as most document-level phenomena rely
on information that is present on the target language side. We evaluate novel
concatenation-based variants where the target context is prepended to the
source language, either in isolation or in combination with the source context.
Experimental results in English-Russian and Basque-Spanish show that including
target context in the source leads to large improvements on target language
phenomena. On source-dependent phenomena, using only target language context in
the source achieves parity with state-of-the-art concatenation approaches, or
slightly underperforms, whereas combining source and target context on the
source side leads to significant gains across the board.

该研究通过拼接语境的方法，研究了在标准文档级神经机器翻译中，是否应进一步提升目标语言的上下文对翻译结果的影响，实验结果表明，在目标语言现象上，将目标语言上下文加入到源语言端能够有显著的提升，而在依赖于源语言的现象方面，将源语言和目标语言的上下文同时加入源语言端能够在所有指标上都有显著的提升。

上下文感知的神经机器翻译中的目标数据推广

Promoting Target Data in Context-aware Neural Machine Translation

Standard machine translation systems process sentences in isolation and hence
ignore extra-sentential information, even though extended context can both
prevent mistakes in ambiguous cases and improve translation coherence. We
introduce a context-aware neural machine translation model designed in such way
that the flow of information from the extended context to the translation model
can be controlled and analyzed. We experiment with an English-Russian subtitles
dataset, and observe that much of what is captured by our model deals with
improving pronoun translation. We measure correspondences between induced
attention distributions and coreference relations and observe that the model
implicitly captures anaphora. It is consistent with gains for sentences where
pronouns need to be gendered in translation. Beside improvements in anaphoric
cases, the model also improves in overall BLEU, both over its context-agnostic
version (+0.7) and over simple concatenation of the context and source
sentences (+0.6).

我们引入了一种上下文感知的神经机器翻译模型，该模型可以控制和分析从扩展上下文到翻译模型的信息流，实验证明该模型改进了代词翻译和在指代方面进行了隐含捕获，而且不仅在指代性情况下有所改进，还提高了整体 BLEU 分数。