Generating vivid and emotional 3D co-speech gestures is crucial for virtual
avatar animation in human-machine interaction applications. While the existing
methods enable generating the gestures to follow a single emotion label, they
overlook that long gesture sequence modeling with emotion transition is more
practical in real scenes. In addition, the lack of large-scale available
datasets with emotional transition speech and corresponding 3D human gestures
also limits the addressing of this task. To fulfill this goal, we first
incorporate the ChatGPT-4 and an audio inpainting approach to construct the
high-fidelity emotion transition human speeches. Considering obtaining the
realistic 3D pose annotations corresponding to the dynamically inpainted
emotion transition audio is extremely difficult, we propose a novel weakly
supervised training strategy to encourage authority gesture transitions.
Specifically, to enhance the coordination of transition gestures w.r.t
different emotional ones, we model the temporal association representation
between two different emotional gesture sequences as style guidance and infuse
it into the transition generation. We further devise an emotion mixture
mechanism that provides weak supervision based on a learnable mixed emotion
label for transition gestures. Last, we present a keyframe sampler to supply
effective initial posture cues in long sequences, enabling us to generate
diverse gestures. Extensive experiments demonstrate that our method outperforms
the state-of-the-art models constructed by adapting single emotion-conditioned
counterparts on our newly defined emotion transition task and datasets.

为了实现逼真的 3D 共语手势，我们提出了一种基于 ChatGPT-4 的弱监督训练策略，通过情感过渡、协调转换手势、混合情感机制以及关键帧抽样等方法，成功地构建了新的情感过渡任务和数据集，并在实验证明了我们的方法优于现有的单一情感条件模型。

弱监督情感转换学习用于多样化的三维配合语言手势生成

Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech  Gesture Generation

Emotion Support Conversation (ESC) is an emerging and challenging task with
the goal of reducing the emotional distress of people. Previous attempts fail
to maintain smooth transitions between utterances in ESC because they ignore to
grasp the fine-grained transition information at each dialogue turn. To solve
this problem, we propose to take into account turn-level state
\textbf{Trans}itions of \textbf{ESC} (\textbf{TransESC}) from three
perspectives, including semantics transition, strategy transition and emotion
transition, to drive the conversation in a smooth and natural way.
Specifically, we construct the state transition graph with a two-step way,
named transit-then-interact, to grasp such three types of turn-level transition
information. Finally, they are injected into the transition-aware decoder to
generate more engaging responses. Both automatic and human evaluations on the
benchmark dataset demonstrate the superiority of TransESC to generate more
smooth and effective supportive responses. Our source code is available at
https://github.com/circle-hit/TransESC.

提出了一种名为 TransESC 的模型，其使用从语义、策略和情感层面的转换信息，从而在情感支持对话中实现转换级别的流畅对话。经自动化和人工评估证明了该模型相比较于之前的尝试，在生成更加流畅和有效的支持式回应方面具有优势。

TransESC：基于转换级别状态的情感支持对话平滑

TransESC: Smoothing Emotional Support Conversation via Turn-Level State  Transition

To provide consistent emotional interaction with users, dialog systems should
be capable to automatically select appropriate emotions for responses like
humans. However, most existing works focus on rendering specified emotions in
responses or empathetically respond to the emotion of users, yet the individual
difference in emotion expression is overlooked. This may lead to inconsistent
emotional expressions and disinterest users. To tackle this issue, we propose
to equip the dialog system with personality and enable it to automatically
select emotions in responses by simulating the emotion transition of humans in
conversation. In detail, the emotion of the dialog system is transitioned from
its preceding emotion in context. The transition is triggered by the preceding
dialog context and affected by the specified personality trait. To achieve
this, we first model the emotion transition in the dialog system as the
variation between the preceding emotion and the response emotion in the
Valence-Arousal-Dominance (VAD) emotion space. Then, we design neural networks
to encode the preceding dialog context and the specified personality traits to
compose the variation. Finally, the emotion for response is selected from the
sum of the preceding emotion and the variation. We construct a dialog dataset
with emotion and personality labels and conduct emotion prediction tasks for
evaluation. Experimental results validate the effectiveness of the
personality-affected emotion transition.

本研究旨在为对话系统引入人格特质，通过模拟人类在对话中的情感转换来自动选择回应中的情感，使用 VAD 情感空间和神经网络模型来呈现情感二元组之间的转换，并验证了此方法的有效性。