Large-scale language models, like ChatGPT, have garnered significant media
attention and stunned the public with their remarkable capacity for generating
coherent text from short natural language prompts. In this paper, we aim to
conduct a systematic inspection of ChatGPT's performance in two controllable
generation tasks, with respect to ChatGPT's ability to adapt its output to
different target audiences (expert vs. layman) and writing styles (formal vs.
informal). Additionally, we evaluate the faithfulness of the generated text,
and compare the model's performance with human-authored texts. Our findings
indicate that the stylistic variations produced by humans are considerably
larger than those demonstrated by ChatGPT, and the generated texts diverge from
human samples in several characteristics, such as the distribution of word
types. Moreover, we observe that ChatGPT sometimes incorporates factual errors
or hallucinations when adapting the text to suit a specific style.

本文研究 ChatGPT 在不同目标受众和写作风格下的表现，发现相比人类撰写的文章，ChatGPT 生成的样本在词汇类型分布等方面存在差异，并可能出现事实错误或幻觉。

ChatGPT 与人类撰写文本：可控文本摘要与句子风格转移的洞见

ChatGPT vs Human-authored Text: Insights into Controllable Text  Summarization and Sentence Style Transfer

Controlling the behavior of language models (LMs) without re-training is a
major open problem in natural language generation. While recent works have
demonstrated successes on controlling simple sentence attributes (e.g.,
sentiment), there has been little progress on complex, fine-grained controls
(e.g., syntactic structure). To address this challenge, we develop a new
non-autoregressive language model based on continuous diffusions that we call
Diffusion-LM. Building upon the recent successes of diffusion models in
continuous domains, Diffusion-LM iteratively denoises a sequence of Gaussian
vectors into word vectors, yielding a sequence of intermediate latent
variables. The continuous, hierarchical nature of these intermediate variables
enables a simple gradient-based algorithm to perform complex, controllable
generation tasks. We demonstrate successful control of Diffusion-LM for six
challenging fine-grained control tasks, significantly outperforming prior work.

本研究提出了一种基于连续扩散的非自回归语言模型 Diffusion-LM，在语言生成的过程中能够通过简单的基于梯度的算法实现复杂的可控生成任务，并成功地应用在六个难度较大的细粒度控制任务之中，表现出优秀的性能。