Addressing the critical shortage of mental health resources for effective
screening, diagnosis, and treatment remains a significant challenge. This
scarcity underscores the need for innovative solutions, particularly in
enhancing the accessibility and efficacy of therapeutic support. Embodied
agents with advanced interactive capabilities emerge as a promising and
cost-effective supplement to traditional caregiving methods. Crucial to these
agents' effectiveness is their ability to simulate non-verbal behaviors, like
backchannels, that are pivotal in establishing rapport and understanding in
therapeutic contexts but remain under-explored. To improve the rapport-building
capabilities of embodied agents we annotated backchannel smiles in videos of
intimate face-to-face conversations over topics such as mental health, illness,
and relationships. We hypothesized that both speaker and listener behaviors
affect the duration and intensity of backchannel smiles. Using cues from speech
prosody and language along with the demographics of the speaker and listener,
we found them to contain significant predictors of the intensity of backchannel
smiles. Based on our findings, we introduce backchannel smile production in
embodied agents as a generation problem. Our attention-based generative model
suggests that listener information offers performance improvements over the
baseline speaker-centric generation approach. Conditioned generation using the
significant predictors of smile intensity provides statistically significant
improvements in empirical measures of generation quality. Our user study by
transferring generated smiles to an embodied agent suggests that agent with
backchannel smiles is perceived to be more human-like and is an attractive
alternative for non-personal conversations over agent without backchannel
smiles.

通过使用基于关键预测因素的注意力生成模型，我们提出在体验式代理人中引入背后微笑产生作为一种生成问题，以改善代理人建立情感联系的能力，并且通过用户研究验证，拥有背后微笑的代理人在非个人对话中被认为更加人性化且更具吸引力。