Understanding the nature of high-quality summaries is crucial to further
improve the performance of multi-document summarization. We propose an approach
to characterize human-written summaries using partial information
decomposition, which decomposes the mutual information provided by all source
documents into union, redundancy, synergy, and unique information. Our
empirical analysis on different MDS datasets shows that there is a direct
dependency between the number of sources and their contribution to the summary.

通过部分信息分解的方法，我们对人工撰写的摘要进行了特征化，发现源文件的数量与其对摘要的贡献存在直接依赖关系。

哪些信息起作用？通过部分信息分解解析人类编写的多文档摘要

Which Information Matters? Dissecting Human-written Multi-document  Summaries with Partial Information Decomposition

While the problem of hallucinations in neural machine translation has long
been recognized, so far the progress on its alleviation is very little. Indeed,
recently it turned out that without artificially encouraging models to
hallucinate, previously existing methods fall short and even the standard
sequence log-probability is more informative. It means that characteristics
internal to the model can give much more information than we expect, and before
using external models and measures, we first need to ask: how far can we go if
we use nothing but the translation model itself ? We propose to use a method
that evaluates the percentage of the source contribution to a generated
translation. Intuitively, hallucinations are translations "detached" from the
source, hence they can be identified by low source contribution. This method
improves detection accuracy for the most severe hallucinations by a factor of 2
and is able to alleviate hallucinations at test time on par with the previous
best approach that relies on external models. Next, if we move away from
internal model characteristics and allow external tools, we show that using
sentence similarity from cross-lingual embeddings further improves these
results.

提出了一种通过评估翻译中源语言贡献百分比的方法，以解决神经机器翻译中出现的幻觉问题，并发现使用跨语言嵌入的句子相似性有助于提高检测精度。

检测和缓解机器翻译中的幻觉：仅基于模型内部工作表现良好，基于句子相似度的方法表现更佳

Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better

Machine Translation systems can produce different types of errors, some of
which get characterized as critical or catastrophic due to the specific
negative impact they can have on users. Automatic or human evaluation metrics
do not necessarily differentiate between such critical errors and more
innocuous ones. In this paper we focus on one type of critical error: added
toxicity. We evaluate and analyze added toxicity when translating a large
evaluation dataset (HOLISTICBIAS, over 472k sentences, covering 13 demographic
axes) from English into 164 languages. The toxicity automatic evaluation shows
that added toxicity across languages varies from 0% to 5%. The output languages
with the most added toxicity tend to be low-resource ones, and the demographic
axes with the most added toxicity include sexual orientation, gender and sex,
and ability. We also perform human evaluation on a subset of 8 directions,
confirming the prevalence of true added toxicity.
We use a measurement of the amount of source contribution to the translation,
where a low source contribution implies hallucination, to interpret what causes
toxicity. We observe that the source contribution is somewhat correlated with
toxicity but that 45.6% of added toxic words have a high source contribution,
suggesting that much of the added toxicity may be due to mistranslations.
Combining the signal of source contribution level with a measurement of
translation robustness allows us to flag 22.3% of added toxicity, suggesting
that added toxicity may be related to both hallucination and the stability of
translations in different contexts. Given these findings, our recommendations
to reduce added toxicity are to curate training data to avoid mistranslations,
mitigate hallucination and check unstable translations.

本文调查了机器翻译中的一种关键性错误：加入有毒信息。作者评估了一个大型数据集的翻译质量，并发现了加入有毒信息的范围和原因。作者认为，为了减少这些错误，需要修改训练数据，减少幻觉，检查不稳定的翻译。