The advent of large language models (LLMs) has made it possible to generate
natural written dialogues between two agents. However, generating human-like
spoken dialogues from these written dialogues remains challenging. Spoken
dialogues have several unique characteristics: they frequently include
backchannels and laughter, and the smoothness of turn-taking significantly
influences the fluidity of conversation. This study proposes CHATS - CHatty
Agents Text-to-Speech - a discrete token-based system designed to generate
spoken dialogues based on written dialogues. Our system can generate speech for
both the speaker side and the listener side simultaneously, using only the
transcription from the speaker side, which eliminates the need for
transcriptions of backchannels or laughter. Moreover, CHATS facilitates natural
turn-taking; it determines the appropriate duration of silence after each
utterance in the absence of overlap, and it initiates the generation of
overlapping speech based on the phoneme sequence of the next utterance in case
of overlap. Experimental evaluations indicate that CHATS outperforms the
text-to-speech baseline, producing spoken dialogues that are more interactive
and fluid while retaining clarity and intelligibility.

该论文描述了 CHATS - CHatty Agents Text-to-Speech，这是一种基于书面对话生成口语对话的离散标记系统，通过仅使用说话方的转录，同时为说话方和倾听方生成语音，消除了对倾听方的转录需求，同时可以促进自然交谈的轮换和流畅对话的生成。

从书面对话向人类般的 AI 代理之间的口语对话生成迈进

Towards human-like spoken dialogue generation between AI agents from  written dialogue

This work focuses on the use of acoustic cues for modeling turn-taking in
dyadic spoken dialogues. Previous work has shown that speaker intentions (e.g.,
asking a question, uttering a backchannel, etc.) can influence turn-taking
behavior and are good predictors of turn-transitions in spoken dialogues.
However, speaker intentions are not readily available for use by automated
systems at run-time; making it difficult to use this information to anticipate
a turn-transition. To this end, we propose a multi-task neural approach for
predicting turn- transitions and speaker intentions simultaneously. Our results
show that adding the auxiliary task of speaker intention prediction improves
the performance of turn-transition prediction in spoken dialogues, without
relying on additional input features during run-time.

本文介绍了使用声学线索模型来理解对话者交替的研究，提出了一种多任务神经方法，同时预测转换转变和发言者意图，实验结果表明，使用发言者意图预测的辅助任务可以提高对话中转换转变的预测性能。