Machine Translation (MT) remains one of the last NLP tasks where large
language models (LLMs) have not yet replaced dedicated supervised systems. This
work exploits the complementary strengths of LLMs and supervised MT by guiding
LLMs to automatically post-edit MT with external feedback on its quality,
derived from Multidimensional Quality Metric (MQM) annotations. Working with
LLaMA-2 models, we consider prompting strategies varying the nature of feedback
provided and then fine-tune the LLM to improve its ability to exploit the
provided guidance. Through experiments on Chinese-English, English-German, and
English-Russian MQM data, we demonstrate that prompting LLMs to post-edit MT
improves TER, BLEU and COMET scores, although the benefits of fine-grained
feedback are not clear. Fine-tuning helps integrate fine-grained feedback more
effectively and further improves translation quality based on both automatic
and human evaluation.

通过使用 Multidimensional Quality Metric (MQM) 注释中的外部反馈来引导大型语言模型 (LLMs) 自动对机器翻译进行后编辑，以提高翻译质量。使用 LLaMA-2 模型，通过改变反馈的特性，我们考虑了不同的提示策略，并对 LLM 进行微调以提高其利用所提供指导的能力，实验证明提示 LLMs 对机器翻译进行后编辑，可以改善 TER、BLEU 和 COMET 分数，微调有助于更有效地整合细粒度的反馈，并基于自动和人工评估进一步提高翻译质量。

使用错误注释引导大型语言模型进行机器翻译

Guiding Large Language Models to Post-Edit Machine Translation with  Error Annotations

Deep learning has led to significant improvement in text summarization with
various methods investigated and improved ROUGE scores reported over the years.
However, gaps still exist between summaries produced by automatic summarizers
and human professionals. Aiming to gain more understanding of summarization
systems with respect to their strengths and limits on a fine-grained syntactic
and semantic level, we consult the Multidimensional Quality Metric(MQM) and
quantify 8 major sources of errors on 10 representative summarization models
manually. Primarily, we find that 1) under similar settings, extractive
summarizers are in general better than their abstractive counterparts thanks to
strength in faithfulness and factual-consistency; 2) milestone techniques such
as copy, coverage and hybrid extractive/abstractive methods do bring specific
improvements but also demonstrate limitations; 3) pre-training techniques, and
in particular sequence-to-sequence pre-training, are highly effective for
improving text summarization, with BART giving the best results.

通过使用多维度质量度量标准（MQM），我们手动量化了 10 种代表性汇总模型中 8 种主要错误来源，发现在相似设置下，提取式汇总器总体上比其抽象式汇总器表现更好，尤其是在忠实度和事实一致性方面。同时，预训练技术，特别是序列到序列的预训练技术，对于提高文本汇总效果非常有效，其中 BART 效果最好。