In this study, we address the importance of modeling behavior style in
virtual agents for personalized human-agent interaction. We propose a machine
learning approach to synthesize gestures, driven by prosodic features and text,
in the style of different speakers, even those unseen during training. Our
model incorporates zero-shot multimodal style transfer using multimodal data
from the PATS database, which contains videos of diverse speakers. We recognize
style as a pervasive element during speech, influencing the expressivity of
communicative behaviors, while content is conveyed through multimodal signals
and text. By disentangling content and style, we directly infer the style
embedding, even for speakers not included in the training phase, without the
need for additional training or fine-tuning. Objective and subjective
evaluations are conducted to validate our approach and compare it against two
baseline methods.

本研究利用机器学习方法进行虚拟代理人的行为风格建模，通过多模态数据进行风格转换，包括不同演讲者的手势合成。通过实验证明，该方法能更好地捕捉语音中的风格信息，不受测试阶段未见演讲者的影响。