Speech-to-text services aim to transcribe input audio as accurately as
possible. They increasingly play a role in everyday life, for example in
personal voice assistants or in customer-company interactions. We evaluate Open
AI's Whisper, a state-of-the-art service outperforming industry competitors.
While many of Whisper's transcriptions were highly accurate, we found that
roughly 1% of audio transcriptions contained entire hallucinated phrases or
sentences, which did not exist in any form in the underlying audio. We
thematically analyze the Whisper-hallucinated content, finding that 38% of
hallucinations include explicit harms such as violence, made up personal
information, or false video-based authority. We further provide hypotheses on
why hallucinations occur, uncovering potential disparities due to speech type
by health status. We call on industry practitioners to ameliorate these
language-model-based hallucinations in Whisper, and to raise awareness of
potential biases in downstream applications of speech-to-text models.

评估了 OpenAI 的 Whisper 语音转文字服务，发现大约 1% 的音频转录中包含了并不存在于音频中的幻觉短语或句子，其中 38% 的幻觉内容包括暴力、虚构的个人信息或虚假的基于视频的权威，并提供了关于幻觉发生原因的假设，揭示了由语音类型和健康状况引起的潜在差异。呼吁业界从业者改善 Whisper 中基于语言模型的幻觉问题，并提高对语音转文字模型在下游应用中潜在偏见的意识。

不慎的耳语：语音转文本幻觉的危害

Careless Whisper: Speech-to-Text Hallucination Harms

Large Language Models (LLMs) have demonstrated remarkable proficiency in
generating fluent text. However, they often encounter the challenge of
generating inaccurate or hallucinated content. This issue is common in both
non-retrieval-based generation and retrieval-augmented generation approaches,
and existing post-hoc rectification methods may not address the accumulated
hallucination errors that may be caused by the "snowballing" issue, especially
in reasoning tasks. To tackle these challenges, we introduce a novel approach
called Real-time Verification and Rectification (Ever). Instead of waiting
until the end of the generation process to rectify hallucinations, Ever employs
a real-time, step-wise generation and hallucination rectification strategy. The
primary objective is to detect and rectify hallucinations as they occur during
the text generation process. When compared to both retrieval-based and
non-retrieval-based baselines, Ever demonstrates a significant improvement in
generating trustworthy and factually accurate text across a diverse range of
tasks, including short-form QA, biography generation, and multi-hop reasoning.

我们介绍了一种名为 Ever 的新方法，通过实时检验和矫正策略，解决大语言模型生成的不准确或虚构内容的问题，并在各种任务中，包括短问题回答、传记生成和多跳推理等方面，相比基于检索和非检索的基准模型，明显提高了生成值得信赖和事实准确的文本的能力。

通过实时验证和修正来减轻大型语言模型中的虚构问题

Ever: Mitigating Hallucination in Large Language Models through  Real-Time Verification and Rectification

Existing large language models (LLMs) are known for generating "hallucinated"
content, namely a fabricated text of plausibly looking, yet unfounded, facts.
To identify when these hallucination scenarios occur, we examine the properties
of the generated text in the embedding space. Specifically, we draw inspiration
from the dynamic mode decomposition (DMD) tool in analyzing the pattern
evolution of text embeddings across sentences. We empirically demonstrate how
the spectrum of sentence embeddings over paragraphs is constantly low-rank for
the generated text, unlike that of the ground-truth text. Importantly, we find
that evaluation cases having LLM hallucinations correspond to ground-truth
embedding patterns with a higher number of modes being poorly approximated by
the few modes associated with LLM embedding patterns. In analogy to near-field
electromagnetic evanescent waves, the embedding DMD eigenmodes of the generated
text with hallucinations vanishes quickly across sentences as opposed to those
of the ground-truth text. This suggests that the hallucinations result from
both the generation techniques and the underlying representation.

通过对文本嵌入空间中生成的文本的属性进行分析，我们发现大语言模型生成的虚构文本模式演化跟真实文本不同，且生成技术和底层表示对产生幻觉有影响。