Large Language Models (LLMs), exemplified by ChatGPT, have significantly
reshaped text generation, particularly in the realm of writing assistance.
While ethical considerations underscore the importance of transparently
acknowledging LLM use, especially in scientific communication, genuine
acknowledgment remains infrequent. A potential avenue to encourage accurate
acknowledging of LLM-assisted writing involves employing automated detectors.
Our evaluation of four cutting-edge LLM-generated text detectors reveals their
suboptimal performance compared to a simple ad-hoc detector designed to
identify abrupt writing style changes around the time of LLM proliferation. We
contend that the development of specialized detectors exclusively dedicated to
LLM-assisted writing detection is necessary. Such detectors could play a
crucial role in fostering more authentic recognition of LLM involvement in
scientific communication, addressing the current challenges in acknowledgment
practices.

利用自动检测器识别大型语言模型辅助写作的研究，发现目前的四种先进的自动生成文本检测器性能不佳，提出专门针对大型语言模型辅助写作检测的专用检测器的发展是必要的。这样的检测器可以在科学交流中促进对大型语言模型参与的真实认可，并解决目前对认可实践的挑战。

检测科学交流中的 LLM 辅助写作：我们已经到达目标了吗？

Detecting LLM-Assisted Writing in Scientific Communication: Are We There  Yet?

We show that the use of large language models (LLMs) is prevalent among crowd
workers, and that targeted mitigation strategies can significantly reduce, but
not eliminate, LLM use. On a text summarization task where workers were not
directed in any way regarding their LLM use, the estimated prevalence of LLM
use was around 30%, but was reduced by about half by asking workers to not use
LLMs and by raising the cost of using them, e.g., by disabling copy-pasting.
Secondary analyses give further insight into LLM use and its prevention: LLM
use yields high-quality but homogeneous responses, which may harm research
concerned with human (rather than model) behavior and degrade future models
trained with crowdsourced data. At the same time, preventing LLM use may be at
odds with obtaining high-quality responses; e.g., when requesting workers not
to use LLMs, summaries contained fewer keywords carrying essential information.
Our estimates will likely change as LLMs increase in popularity or
capabilities, and as norms around their usage change. Yet, understanding the
co-evolution of LLM-based tools and users is key to maintaining the validity of
research done using crowdsourcing, and we provide a critical baseline before
widespread adoption ensues.

我们的研究显示，在人群工作者中普遍使用大型语言模型 (LLM)，而有针对性的缓解策略可以显著降低 LLM 的使用，但不能完全消除。在一个文本摘要任务中，未对工作者在任何方面指示 LLM 的使用，估计 LLM 的使用普遍程度约为 30％，但通过要求工作者不使用 LLM 和提高使用 LLM 的代价（例如禁用复制粘贴）减少了约一半。副本分析进一步揭示了 LLM 的使用及其预防的见解：LLM 的使用产生高质量但同质化的回答，可能会损害关注人类（而非模型）行为的研究并降低以众包数据进行训练的未来模型的质量。同时，防止 LLM 的使用可能与获取高质量回答的目标相悖；例如，要求工作者不使用 LLM 会导致摘要中的关键字数量减少，从而丢失了关键信息。我们的估计可能随着 LLM 的普及或能力的增加以及其使用方式的变化而改变。然而，在广泛采用之前，了解基于 LLM 的工具和用户的共同演变对于维护使用众包进行的研究的有效性至关重要，我们提供了一个重要的基线。