Laughter is a unique expression, essential to affirmative social interactions
of humans. Although current 3D talking head generation methods produce
convincing verbal articulations, they often fail to capture the vitality and
subtleties of laughter and smiles despite their importance in social context.
In this paper, we introduce a novel task to generate 3D talking heads capable
of both articulate speech and authentic laughter. Our newly curated dataset
comprises 2D laughing videos paired with pseudo-annotated and human-validated
3D FLAME parameters and vertices. Given our proposed dataset, we present a
strong baseline with a two-stage training scheme: the model first learns to
talk and then acquires the ability to express laughter. Extensive experiments
demonstrate that our method performs favorably compared to existing approaches
in both talking head generation and expressing laughter signals. We further
explore potential applications on top of our proposed method for rigging
realistic avatars.

本文介绍了一种生成 3D 说话头部并能够表达真实笑声的新方法，通过 2D 视频和 3D 参数数据集的配对，展示了一种基于两阶段训练方案的强大基线模型，该模型在说话头部生成和表达笑声信号方面相比现有方法表现出优越性，并探索了真实化角色控制的潜在应用。

笑话语：用笑声生成表情豐富的三维说话头部

LaughTalk: Expressive 3D Talking Head Generation with Laughter

This paper presents a novel approach for generating 3D talking heads from raw
audio inputs. Our method grounds on the idea that speech related movements can
be comprehensively and efficiently described by the motion of a few control
points located on the movable parts of the face, i.e., landmarks. The
underlying musculoskeletal structure then allows us to learn how their motion
influences the geometrical deformations of the whole face. The proposed method
employs two distinct models to this aim: the first one learns to generate the
motion of a sparse set of landmarks from the given audio. The second model
expands such landmarks motion to a dense motion field, which is utilized to
animate a given 3D mesh in neutral state. Additionally, we introduce a novel
loss function, named Cosine Loss, which minimizes the angle between the
generated motion vectors and the ground truth ones. Using landmarks in 3D
talking head generation offers various advantages such as consistency,
reliability, and obviating the need for manual-annotation. Our approach is
designed to be identity-agnostic, enabling high-quality facial animations for
any users without additional data or training.

本篇研究提出了一种新方法，通过音频输入生成 3D 说话人头部动画，并利用面部的传动部位上的控制点来描述语音相关的运动，并利用两个不同的模型来实现；该方法具有身份不相关性，可实现任何用户的高质量面部动画。利用陆标在 3D 说话人头部动画生成中提供了各种优点，例如一致性，可靠性和不需要手动注释。