Coherence is an essential property of well-written texts, that refers to the
way textual units relate to one another. In the era of generative AI, coherence
assessment is essential for many NLP tasks; summarization, generation,
long-form question-answering, and more. However, in NLP {coherence} is an
ill-defined notion, not having a formal definition or evaluation metrics, that
would allow for large-scale automatic and systematic coherence assessment. To
bridge this gap, in this work we employ the formal linguistic definition of
\citet{Reinhart:1980} of what makes a discourse coherent, consisting of three
conditions -- {\em cohesion, consistency} and {\em relevance} -- and formalize
these conditions as respective computational tasks. We hypothesize that (i) a
model trained on all of these tasks will learn the features required for
coherence detection, and that (ii) a joint model for all tasks will exceed the
performance of models trained on each task individually. On two benchmarks for
coherence scoring rated by humans, one containing 500 automatically-generated
short stories and another containing 4k real-world texts, our experiments
confirm that jointly training on the proposed tasks leads to better performance
on each task compared with task-specific models, and to better performance on
assessing coherence overall, compared with strong baselines. We conclude that
the formal and computational setup of coherence as proposed here provides a
solid foundation for advanced methods of large-scale automatic assessment of
coherence.

该论文提出了一种正式和计算的口头表达的一致性的设置和评估方法，该方法通过联合训练多种任务的模型，在评估一致性的各个任务上表现出更好的性能，为大规模自动评估一致性提供了坚实的基础。

自动一致性评估的新型计算和建模基础

A Novel Computational and Modeling Foundation for Automatic Coherence  Assessment

Unlike well-structured text, such as news reports and encyclopedia articles,
dialogue content often comes from two or more interlocutors, exchanging
information with each other. In such a scenario, the topic of a conversation
can vary upon progression and the key information for a certain topic is often
scattered across multiple utterances of different speakers, which poses
challenges to abstractly summarize dialogues. To capture the various topic
information of a conversation and outline salient facts for the captured
topics, this work proposes two topic-aware contrastive learning objectives,
namely coherence detection and sub-summary generation objectives, which are
expected to implicitly model the topic change and handle information scattering
challenges for the dialogue summarization task. The proposed contrastive
objectives are framed as auxiliary tasks for the primary dialogue summarization
task, united via an alternative parameter updating strategy. Extensive
experiments on benchmark datasets demonstrate that the proposed simple method
significantly outperforms strong baselines and achieves new state-of-the-art
performance. The code and trained models are publicly available via
\href{https://github.com/Junpliu/ConDigSum}{this https URL}.

通过引入一种叫做对比学习的技术来解决对话总结中信息分散和主题变化的挑战，该方法使用了两个具体的辅助任务 (coherence detection 和 sub-summary generation)，在基于原始对话总结任务上取得了最优性能，显著超过强基准模型。