Text summarization and simplification are among the most widely used
applications of AI. However, models developed for such tasks are often prone to
hallucination, which can result from training on unaligned data. One efficient
approach to address this issue is Loss Truncation (LT) (Kang and Hashimoto,
2020), an approach to modify the standard log loss to adaptively remove noisy
examples during training. However, we find that LT alone yields a considerable
number of hallucinated entities on various datasets. We study the behavior of
the underlying losses between factual and non-factual examples, to understand
and refine the performance of LT. We demonstrate that LT's performance is
limited when the underlying assumption that noisy targets have higher NLL loss
is not satisfied, and find that word-level NLL among entities provides better
signal for distinguishing factuality. We then leverage this to propose a
fine-grained NLL loss and fine-grained data cleaning strategies, and observe
improvements in hallucination reduction across some datasets. Our work is
available at https://this https URL

文本摘要和简化是人工智能最常用的应用之一，然而，针对此类任务开发的模型往往容易出现幻觉，这可能是由于在不对齐的数据上进行训练。为了解决这个问题，我们提出了一种有效的方法，即损失截断（Loss Truncation）（Kang and Hashimoto，2020），通过修改标准的对数损失来自适应性地删除训练中的噪声示例。然而，我们发现单独使用损失截断在各种数据集上会产生相当多的幻觉实体。我们研究了真实和非真实示例之间潜在损失的行为，以理解和改进损失截断的性能。我们证明当噪声目标的 NLL 损失不如预期时，损失截断的性能会受到限制，并且发现实体之间的词级 NLL 提供更好的区分真实性的信号。然后我们利用此结果提出了一种细粒度的 NLL 损失和细粒度的数据清洗策略，观察到在某些数据集上幻觉减少的改进。

关于细粒度损失截断的好处：一个关于摘要中事实性的案例研究

On the Benefits of Fine-Grained Loss Truncation: A Case Study on  Factuality in Summarization

Neural language models are usually trained to match the distributional
properties of a large-scale corpus by minimizing the log loss. While
straightforward to optimize, this approach forces the model to reproduce all
variations in the dataset, including noisy and invalid references (e.g.,
misannotation and hallucinated facts). Worse, the commonly used log loss is
overly sensitive to such phenomena and even a small fraction of noisy data can
degrade performance. In this work, we show that the distinguishability of the
models and reference serves as a principled and robust alternative for handling
invalid references. To optimize distinguishability, we propose loss truncation,
which adaptively removes high loss examples during training. We show this is as
easy to optimize as log loss and tightly bounds distinguishability under noise.
Empirically, we demonstrate that loss truncation outperforms existing baselines
on distinguishability on a summarization task, and show that samples generated
by the loss truncation model have factual accuracy ratings that exceed those of
baselines and match human references.

本文提出了一种利用模型与参考之间的可区分性来处理无效参考的算法，称为损失截断。与常用的 log 损失相比，该方法可以更好地应对嘈杂和无效的参考，并取得了更好的实验效果，生成的样本拥有超越基线并与人类参考相匹配的事实准确性评分。