We present our latest findings on backchannel modeling novelly motivated by the canonical use of the minimal responses Yeah and Uh-huh in English and their correspondent tokens in German, and the effect of encoding the speaker-listener interaction. Backchanneling theories emphasize the active and continuous role of the listener in the course of the conversation, their effects on the speaker's subsequent talk, and the consequent dynamic speaker-listener interaction. Therefore, we propose a neural-based acoustic backchannel classifier on minimal responses by processing acoustic features from the speaker speech, capturing and imitating listeners' backchanneling behavior, and encoding speaker-listener interaction. Our experimental results on the Switchboard and GECO datasets reveal that in almost all tested scenarios the speaker or listener behavior embeddings help the model make more accurate backchannel predictions. More importantly, a proper interaction encoding strategy, i.e., combining the speaker and listener embeddings, leads to the best performance on both datasets in terms of F1-score.

本文提出一种新颖的后发模型，模拟了英语中 Yeah 和 Uh-huh 以及德语中的对应标记的最小响应的使用，通过处理讲话者语音的声学特征，捕捉和模仿听众的后发行为，并编码演讲者-听众交互，提出了一种基于神经网络的声学后发分类器。通过在Switchboard和GECO数据集上的实验，我们的研究表明，在几乎所有测试场景中，演讲者或听众行为嵌入有助于模型进行更准确的后发预测。更重要的是，一种适当的交互编码策略，即结合演讲者和听众嵌入，能够在F1-score方面在两个数据集上实现最佳性能。

为反馈预测建立演讲者 - 听众交互模型