Collaborative robots must effectively communicate their internal state to
humans to enable a smooth interaction. Nonverbal communication is widely used
to communicate information during human-robot interaction, however, such
methods may also be misunderstood, leading to communication errors. In this
work, we explore modulating the acoustic parameter values (pitch bend, beats
per minute, beats per loop) of nonverbal auditory expressions to convey
functional robot states (accomplished, progressing, stuck). We propose a
reinforcement learning (RL) algorithm based on noisy human feedback to produce
accurately interpreted nonverbal auditory expressions. The proposed approach
was evaluated through a user study with 24 participants. The results
demonstrate that: 1. Our proposed RL-based approach is able to learn suitable
acoustic parameter values which improve the users' ability to correctly
identify the state of the robot. 2. Algorithm initialization informed by
previous user data can be used to significantly speed up the learning process.
3. The method used for algorithm initialization strongly influences whether
participants converge to similar sounds for each robot state. 4. Modulation of
pitch bend has the largest influence on user association between sounds and
robotic states.

通过调节非语言音频表达的声学参数值，基于强化学习算法和人类反馈，研究评估了协作机器人内部状态的非语言音频表达对用户是否准确理解机器人状态的影响。

学习用非语言表达传达功能状态，以提高人机协作

Learning to Communicate Functional States with Nonverbal Expressions for  Improved Human-Robot Collaboration

Non verbal behaviours such as gestures, facial expressions, body posture, and
para-linguistic cues have been shown to complement or clarify verbal messages.
Hence to improve telepresence, in form of an avatar, it is important to model
these behaviours, especially in dyadic interactions. Creating such personalized
avatars not only requires to model intrapersonal dynamics between a avatar's
speech and their body pose, but it also needs to model interpersonal dynamics
with the interlocutor present in the conversation. In this paper, we introduce
a neural architecture named Dyadic Residual-Attention Model (DRAM), which
integrates intrapersonal (monadic) and interpersonal (dyadic) dynamics using
selective attention to generate sequences of body pose conditioned on audio and
body pose of the interlocutor and audio of the human operating the avatar. We
evaluate our proposed model on dyadic conversational data consisting of pose
and audio of both participants, confirming the importance of adaptive attention
between monadic and dyadic dynamics when predicting avatar pose. We also
conduct a user study to analyze judgments of human observers. Our results
confirm that the generated body pose is more natural, models intrapersonal
dynamics and interpersonal dynamics better than non-adaptive monadic/dyadic
models.

本文提出了一种名为 Dyadic Residual-Attention Model（DRAM）的神经网络结构，通过选择性注意力整合单向和双向动态，以生成人物姿势序列，从而改善虚拟代表（avatar）的远程存在感，进而在双向交流中展现出更自然的人物姿势，并进行实验验证和用户研究，证实该模型在预测 avatar 姿势方面比非自适应的单向 / 双向模型更有效，更逼真。

是否使用 React：面向双人对话的个性化化身姿态预测

To React or not to React: End-to-End Visual Pose Forecasting for  Personalized Avatar during Dyadic Conversations

Human nonverbal emotional communication in dyadic dialogs is a process of
mutual influence and adaptation. Identifying the direction of influence, or
cause-effect relation between participants is a challenging task, due to two
main obstacles. First, distinct emotions might not be clearly visible. Second,
participants cause-effect relation is transient and variant over time. In this
paper, we address these difficulties by using facial expressions that can be
present even when strong distinct facial emotions are not visible. We also
propose to apply a relevant interval selection approach prior to causal
inference to identify those transient intervals where adaptation process
occurs. To identify the direction of influence, we apply the concept of Granger
causality to the time series of facial expressions on the set of relevant
intervals. We tested our approach on synthetic data and then applied it to
newly, experimentally obtained data. Here, we were able to show that a more
sensitive facial expression detection algorithm and a relevant interval
detection approach is most promising to reveal the cause-effect pattern for
dyadic communication in various instructed interaction conditions.

本文提出一种新的面部表情检测算法以及区间选择方法，用于识别参与者在语言以外的情境中在面部表情方面的互相影响和适应过程。通过对一系列相关区间的面部表情时间序列应用 Granger 因果，我们成功地揭示了在不同的实验条件下双方交流的因果模式。