Recent Generative Artificial Intelligence (GenAI) trends focus on various
applications, including creating stories, illustrations, poems, articles,
computer code, music compositions, and videos. Extrinsic hallucinations are a
critical limitation of such GenAI, which can lead to significant challenges in
achieving and maintaining the trustworthiness of GenAI. In this paper, we
propose two new concepts that we believe will aid the research community in
addressing limitations associated with the application of GenAI models. First,
we propose a definition for the "desirability" of GenAI outputs and three
factors which are observed to influence it. Second, drawing inspiration from
Martin Fowler's code smells, we propose the concept of "prompt smells" and the
adverse effects they are observed to have on the desirability of GenAI outputs.
We expect our work will contribute to the ongoing conversation about the
desirability of GenAI outputs and help advance the field in a meaningful way.

近期的生成人工智能 (GenAI) 趋势关注了各种应用，包括创作故事、插图、诗歌、文章、计算机代码、音乐作品和视频。本文提出了两个新概念，旨在帮助研究界解决与应用 GenAI 模型相关的限制问题。首先，我们提出了 GenAI 输出的 “可取性” 的定义以及观察到影响其可取性的三个因素；其次，我们借鉴了 Martin Fowler 的 “代码异味” 思想，提出了 “提示异味” 的概念及其对 GenAI 输出可取性的不良影响。我们期望这项工作能为关于 GenAI 输出可取性的持续讨论做出贡献并推动该领域的进展。

提示異味：不良生成 AI 輸出的前兆

Prompt Smells: An Omen for Undesirable Generative AI Outputs

How well can large language models (LLMs) generate summaries? We develop new
datasets and conduct human evaluation experiments to evaluate the zero-shot
generation capability of LLMs across five distinct summarization tasks. Our
findings indicate a clear preference among human evaluators for LLM-generated
summaries over human-written summaries and summaries generated by fine-tuned
models. Specifically, LLM-generated summaries exhibit better factual
consistency and fewer instances of extrinsic hallucinations. Due to the
satisfactory performance of LLMs in summarization tasks (even surpassing the
benchmark of reference summaries), we believe that most conventional works in
the field of text summarization are no longer necessary in the era of LLMs.
However, we recognize that there are still some directions worth exploring,
such as the creation of novel datasets with higher quality and more reliable
evaluation methods.

大型语言模型在总结任务中表现出令人满意的性能，超过了参考摘要的基准，人类评估者明显偏好大型语言模型生成的摘要而不是人工撰写的摘要和经过微调的模型生成的摘要，因为大型语言模型生成的摘要具有更好的事实连贯性和更少的外在幻觉实例。