Imbuing machines with the ability to talk has been a longtime pursuit of
artificial intelligence (AI) research. From the very beginning, the community
has not only aimed to synthesise high-fidelity speech that accurately conveys
the semantic meaning of an utterance, but also to colour it with inflections
that cover the same range of affective expressions that humans are capable of.
After many years of research, it appears that we are on the cusp of achieving
this when it comes to single, isolated utterances. This unveils an abundance of
potential avenues to explore when it comes to combining these single utterances
with the aim of synthesising more complex, longer-term behaviours. In the
present chapter, we outline the methodological advances that brought us so far
and sketch out the ongoing efforts to reach that coveted next level of
artificial expressivity. We also discuss the societal implications coupled with
rapidly advancing expressive speech synthesis (ESS) technology and highlight
ways to mitigate those risks and ensure the alignment of ESS capabilities with
ethical norms.

将机器赋予说话的能力一直是人工智能研究的长期追求。通过对语义意义和情感表达范围进行准确合成，我们正处在实现这一目标的边缘。本文概述了让我们取得如此成果的方法学进展，并勾勒了达到人工表达能力的下一个层次的不断努力。同时，我们还探讨了快速发展的表达性语音合成（ESS）技术的社会影响以及降低风险和确保 ESS 能力与道德规范对齐的方法。

表达能力与语音合成

Expressivity and Speech Synthesis

Although affective expressions of individuals have been extensively studied
using social media, research has primarily focused on the Western context.
There are substantial differences among cultures that contribute to their
affective expressions. This paper examines the differences between Twitter (X)
in the United States and Sina Weibo posts in China on two primary dimensions of
affect - valence and arousal. We study the difference in the functional
relationship between arousal and valence (so-called V-shaped) among individuals
in the US and China and explore the associated content differences.
Furthermore, we correlate word usage and topics in both platforms to interpret
their differences. We observe that for Twitter users, the variation in
emotional intensity is less distinct between negative and positive emotions
compared to Weibo users, and there is a sharper escalation in arousal
corresponding with heightened emotions. From language features, we discover
that affective expressions are associated with personal life and feelings on
Twitter, while on Weibo such discussions are about socio-political topics in
the society. These results suggest a West-East difference in the V-shaped
relationship between valence and arousal of affective expressions on social
media influenced by content differences. Our findings have implications for
applications and theories related to cultural differences in affective
expressions.

通过对美国的 Twitter 和中国的新浪微博的比较研究，本文探讨了不同社交媒体平台上用户情感（两个主要维度为情感价值和情感唤起度）之间的差异，同时也研究了其关联的内容差异，并发现来自不同文化背景的用户对情感的表达方式存在明显差异，并从语言特征上解释了这些差异。

美国和中国之间的语言基于的情绪价值和唤起表达：跨文化研究

Language-based Valence and Arousal Expressions between the United States  and China: a Cross-Cultural Examination

We present a generative adversarial network to synthesize 3D pose sequences
of co-speech upper-body gestures with appropriate affective expressions. Our
network consists of two components: a generator to synthesize gestures from a
joint embedding space of features encoded from the input speech and the seed
poses, and a discriminator to distinguish between the synthesized pose
sequences and real 3D pose sequences. We leverage the Mel-frequency cepstral
coefficients and the text transcript computed from the input speech in separate
encoders in our generator to learn the desired sentiments and the associated
affective cues. We design an affective encoder using multi-scale
spatial-temporal graph convolutions to transform 3D pose sequences into latent,
pose-based affective features. We use our affective encoder in both our
generator, where it learns affective features from the seed poses to guide the
gesture synthesis, and our discriminator, where it enforces the synthesized
gestures to contain the appropriate affective expressions. We perform extensive
evaluations on two benchmark datasets for gesture synthesis from the speech,
the TED Gesture Dataset and the GENEA Challenge 2020 Dataset. Compared to the
best baselines, we improve the mean absolute joint error by 10--33%, the mean
acceleration difference by 8--58%, and the Fr\'echet Gesture Distance by
21--34%. We also conduct a user study and observe that compared to the best
current baselines, around 15.28% of participants indicated our synthesized
gestures appear more plausible, and around 16.32% of participants felt the
gestures had more appropriate affective expressions aligned with the speech.

本文介绍了一种基于生成对抗网络的 3D 姿势序列综合方法，通过合适的情感表达来综合共语上半身手势姿势。