Large language models (LLMs) are becoming a one-fits-many solution, but they
sometimes hallucinate or produce unreliable output. In this paper, we
investigate how hypothesis ensembling can improve the quality of the generated
text for the specific problem of LLM-based machine translation. We experiment
with several techniques for ensembling hypotheses produced by LLMs such as
ChatGPT, LLaMA, and Alpaca. We provide a comprehensive study along multiple
dimensions, including the method to generate hypotheses (multiple prompts,
temperature-based sampling, and beam search) and the strategy to produce the
final translation (instruction-based, quality-based reranking, and minimum
Bayes risk (MBR) decoding). Our results show that MBR decoding is a very
effective method, that translation quality can be improved using a small number
of samples, and that instruction tuning has a strong impact on the relation
between the diversity of the hypotheses and the sampling temperature.

通过对多个大型语言模型 (LLMs) 进行假设集成的研究，对于基于 LLM 的机器翻译问题，我们探讨了如何提高生成文本的质量。我们尝试了多种假设集成技术，如 ChatGPT、LLaMA 和 Alpaca，并进行了综合研究，包括生成假设的方法（多个提示、温度采样和束搜索）以及生成最终翻译的策略（基于指令、基于质量的重新排序和最小贝叶斯风险解码）。我们的研究结果表明，MBR 解码是一种非常有效的方法，使用少量样本可以提高翻译质量，指令调整对假设的多样性和采样温度之间的关系具有很大影响。

使用大型语言模型进行翻译假设集成的实证研究

An Empirical Study of Translation Hypothesis Ensembling with Large  Language Models

Traditional neural machine translation (NMT) systems often fail to translate
sentences that contain culturally specific information. Most previous NMT
methods have incorporated external cultural knowledge during training, which
requires fine-tuning on low-frequency items specific to the culture. Recent
in-context learning utilizes lightweight prompts to guide large language models
(LLMs) to perform machine translation, however, whether such an approach works
in terms of injecting culture awareness into machine translation remains
unclear. To this end, we introduce a new data curation pipeline to construct a
culturally relevant parallel corpus, enriched with annotations of
cultural-specific entities. Additionally, we design simple but effective
prompting strategies to assist this LLM-based translation. Extensive
experiments show that our approaches can largely help incorporate cultural
knowledge into LLM-based machine translation, outperforming traditional NMT
systems in translating cultural-specific sentences.

本研究着眼于如何将文化知识应用于大型语言模型的机器翻译中，提出了一种新的数据筛选方法来构建具有文化相关性的平行语料库，并设计了简单有效的提示策略以帮助大型语言模型实现机器翻译。经过广泛实验表明，这些方法可以大大帮助将文化知识融入到大型语言模型的机器翻译中，从而在翻译文化特定的句子方面优于传统的 NMT 系统。