Conversation demands attention. Speakers must call words to mind, listeners
must make sense of them, and both together must negotiate this flow of
information, all in fractions of a second. We used large language models to
study how this works in a large-scale dataset of English-language conversation,
the CANDOR corpus. We provide a new estimate of the information density of
unstructured conversation, of approximately 13 bits/second, and find
significant effects associated with the cognitive load of both retrieving, and
presenting, that information. We also reveal a role for backchannels -- the
brief yeahs, uh-huhs, and mhmms that listeners provide -- in regulating the
production of novelty: the lead-up to a backchannel is associated with
declining information rate, while speech downstream rebounds to previous rates.
Our results provide new insights into long-standing theories of how we respond
to fluctuating demands on cognitive resources, and how we negotiate those
demands in partnership with others.

使用大型语言模型研究英语对话的大规模数据集，提供了关于非结构化对话信息密度的新估计，以及涉及检索和呈现信息的认知负荷的显著效应，同时揭示了后导评价词在调节新颖性产生中的作用。我们的研究结果为我们如何应对认知资源波动的长期理论和与他人合作应对这些需求的方式提供了新的见解。

从 “嗯” 到 “是的”：人类对话中的信息生成、预测和调控

From "um" to "yeah": Producing, predicting, and regulating information  flow in human conversation

Listeners use short interjections, so-called backchannels, to signify
attention or express agreement. The automatic analysis of this behavior is of
key importance for human conversation analysis and interactive conversational
agents. Current state-of-the-art approaches for backchannel analysis from
visual behavior make use of two types of features: features based on body pose
and features based on facial behavior. At the same time, transformer neural
networks have been established as an effective means to fuse input from
different data sources, but they have not yet been applied to backchannel
analysis. In this work, we conduct a comprehensive evaluation of multi-modal
transformer architectures for automatic backchannel analysis based on pose and
facial information. We address both the detection of backchannels as well as
the task of estimating the agreement expressed in a backchannel. In evaluations
on the MultiMediate'22 backchannel detection challenge, we reach 66.4% accuracy
with a one-layer transformer architecture, outperforming the previous state of
the art. With a two-layer transformer architecture, we furthermore set a new
state of the art (0.0604 MSE) on the task of estimating the amount of agreement
expressed in a backchannel.

本文研究使用多模态 transformer 架构，基于身体姿势和面部信息进行自动后向反馈分析的目标检测和表达同意的任务，达到了 66.4％的检测准确率，并在表达同意的任务上创造了新的最佳成果（0.0604 MSE）。

利用 Transformer 网络从视频中检测 Backchannel 并估算一致性

Backchannel Detection and Agreement Estimation from Video with  Transformer Networks

In recent years, many works have investigated the feasibility of
conversational robots for performing specific tasks, such as healthcare and
interview. Along with this development comes a practical issue: how should we
synthesize robotic voices to meet the needs of different situations? In this
paper, we discuss this issue from three perspectives: 1) the difficulties of
synthesizing non-verbal and interaction-oriented speech signals, particularly
backchannels; 2) the scenario classification for robotic voice synthesis; 3)
the ethical issues regarding the design of robot voice for its emotion and
identity. We present the findings of relevant literature and our prior work,
trying to bring the attention of human-robot interaction researchers to design
better conversational robots in the future.

本文从非语言和互动语音信号的综合、机器人语音综合的情景分类和机器人语音情感和认同的设计三个方面来讨论优化人机交互的通讯机器人设计。