Despite the recent advances in applying pre-trained language models to
generate high-quality texts, generating long passages that maintain long-range
coherence is yet challenging for these models. In this paper, we propose
DiscoDVT, a discourse-aware discrete variational Transformer to tackle the
incoherence issue. DiscoDVT learns a discrete variable sequence that summarizes
the global structure of the text and then applies it to guide the generation
process at each decoding step. To further embed discourse-aware information
into the discrete latent representations, we introduce an auxiliary objective
to model the discourse relations within the text. We conduct extensive
experiments on two open story generation datasets and demonstrate that the
latent codes learn meaningful correspondence to the discourse structures that
guide the model to generate long texts with better long-range coherence.

本文提出 DiscoDVT：一种基于离散变分 Transformer 的话语感知模型，用于帮助预训练语言模型生成保持长距离连贯性的长文章。通过学习一个可以概括全文结构的离散变量序列，并利用其指导每个解码步骤的生成过程。试验表明，离散编码与话语结构有显著联系，提高长段连贯性。

DiscoDVT: 使用意识到语篇的离散变分变换器生成长文本

DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational  Transformer

Developers of text generation models rely on automated evaluation metrics as
a stand-in for slow and expensive manual evaluations. However, image captioning
metrics have struggled to give accurate learned estimates of the semantic and
pragmatic success of output text. We address this weakness by introducing the
first discourse-aware learned generation metric for evaluating image
descriptions. Our approach is inspired by computational theories of discourse
for capturing information goals using coherence. We present a dataset of
image$\unicode{x2013}$description pairs annotated with coherence relations. We
then train a coherence-aware metric on a subset of the Conceptual Captions
dataset and measure its effectiveness$\unicode{x2014}$its ability to predict
human ratings of output captions$\unicode{x2014}$on a test set composed of
out-of-domain images. We demonstrate a higher Kendall Correlation Coefficient
for our proposed metric with the human judgments for the results of a number of
state-of-the-art coherence-aware caption generation models when compared to
several other metrics including recently proposed learned metrics such as
BLEURT and BERTScore.

本研究提出了一种基于计算话语理论的生成度量，用于评估图像描述生成模型的语义和语用成功，与最近提出的学习指标相比，在人类评分预测方面表现更好。

COSMic: 一个针对图像描述的一致性感知生成度量

COSMic: A Coherence-Aware Generation Metric for Image Descriptions

Zero pronouns (ZPs) are frequently omitted in pro-drop languages, but should
be recalled in non-pro-drop languages. This discourse phenomenon poses a
significant challenge for machine translation (MT) when translating texts from
pro-drop to non-pro-drop languages. In this paper, we propose a unified and
discourse-aware ZP translation approach for neural MT models. Specifically, we
jointly learn to predict and translate ZPs in an end-to-end manner, allowing
both components to interact with each other. In addition, we employ
hierarchical neural networks to exploit discourse-level context, which is
beneficial for ZP prediction and thus translation. Experimental results on both
Chinese-English and Japanese-English data show that our approach significantly
and accumulatively improves both translation performance and ZP prediction
accuracy over not only baseline but also previous works using external ZP
prediction models. Extensive analyses confirm that the performance improvement
comes from the alleviation of different kinds of errors especially caused by
subjective ZPs.

本文提出了一种统一且考虑话语的零代词（Zero pronouns, ZPs）翻译方法，利用神经网络实现对 ZP 的预测和翻译，并采用分层神经网络来利用话语上下文，实验结果表明该方法在中英和日英语料数据上显著提高了机器翻译和 ZP 预测的准确性，尤其是缓解了主观 ZPs 带来的误差。