Human speech is often accompanied by body gestures including arm and hand
gestures. We present a method that reenacts a high-quality video with gestures
matching a target speech audio. The key idea of our method is to split and
re-assemble clips from a reference video through a novel video motion graph
encoding valid transitions between clips. To seamlessly connect different clips
in the reenactment, we propose a pose-aware video blending network which
synthesizes video frames around the stitched frames between two clips.
Moreover, we developed an audio-based gesture searching algorithm to find the
optimal order of the reenacted frames. Our system generates reenactments that
are consistent with both the audio rhythms and the speech content. We evaluate
our synthesized video quality quantitatively, qualitatively, and with user
studies, demonstrating that our method produces videos of much higher quality
and consistency with the target audio compared to previous work and baselines.

该论文提供了一种利用语音内容匹配身体动作的视频再现方法，通过剪辑、组合、动作建模、音频节奏搜索等技术，实现了更高质量和一致性的视频合成。

基于音频的神经手势再现与视频运动图的应用

Audio-driven Neural Gesture Reenactment with Video Motion Graphs

Automatic emotion recognition has become a trending research topic in the
past decade. While works based on facial expressions or speech abound,
recognizing affect from body gestures remains a less explored topic. We present
a new comprehensive survey hoping to boost research in the field. We first
introduce emotional body gestures as a component of what is commonly known as
"body language" and comment general aspects as gender differences and culture
dependence. We then define a complete framework for automatic emotional body
gesture recognition. We introduce person detection and comment static and
dynamic body pose estimation methods both in RGB and 3D. We then comment the
recent literature related to representation learning and emotion recognition
from images of emotionally expressive gestures. We also discuss multi-modal
approaches that combine speech or face with body gestures for improved emotion
recognition. While pre-processing methodologies (e.g. human detection and pose
estimation) are nowadays mature technologies fully developed for robust large
scale analysis, we show that for emotion recognition the quantity of labelled
data is scarce, there is no agreement on clearly defined output spaces and the
representations are shallow and largely based on naive geometrical
representations.

本文调研了自动情感识别领域中人体动作的情感表达。提出了完整的框架，包括人体检测、姿态估计、情感识别等组成部分。并讨论了多模态情感表达识别的方法。