Animating an object in 3D often requires an articulated structure, e.g. a
kinematic chain or skeleton of the manipulated object with proper skinning
weights, to obtain smooth movements and surface deformations. However, existing
models that allow direct pose manipulations are either limited to specific
object categories or built with specialized equipment. To reduce the work
needed for creating animatable 3D models, we propose a novel reconstruction
method that learns an animatable kinematic chain for any articulated object.
Our method operates on monocular videos without prior knowledge of the object's
shape or underlying structure. Our approach is on par with state-of-the-art 3D
surface reconstruction methods on various articulated object categories while
enabling direct pose manipulations by re-posing the learned kinematic chain.

提出一种新的重建方法，可以学习任何物品的动画运动学链，不需要事先了解物品的形状或基础结构，能够实现直接姿势操纵，并在各种复杂的关节对象类型上获得与最先进的三维表面重建方法相当的表现。

从单目视频中构建类别不可知和可动画的三维模型：CAMM

CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular  Videos

This work proposes an interactive art installation "Mood spRing" designed to
reflect the mood of the environment through interpretation of language and
tone. Mood spRing consists of an AI program that controls an immersive 3D
animation of the seasons. If the AI program perceives the language and tone of
the users as pleasant, the animation progresses through idealized renditions of
seasons. Otherwise, it slips into unpleasant weather and natural disasters of
the season. To interpret the language and tone of the user interaction, hybrid
state-of-the-art emotion detection methods are applied to the user audio and
text inputs. The emotional states detected separately from tone and language
are fused by a novel approach that aims at minimizing the possible model
disparity across diverse demographic groups.

本文提出了一种交互式艺术装置 “Mood spRing”，它通过解释语言和语调来反映环境的心情。 Mood spRing 由控制季节沉浸式 3D 动画的人工智能程序组成。如果 AI 程序感知到用户的语言和语调是愉快的，则动画将通过理想化的季节描绘继续进行。否则，它将陷入不愉快的天气和自然灾害中。通过将用户音频和文本输入应用混合现代情感检测方法来解释用户互动的语言和语调情绪状态，通过新方法融合从音调和语言中分别检测到的情感状态，旨在最大程度地降低各种人口群体可能的模型差异。

用户交互情绪状态的视觉反馈

Visual Response to Emotional State of User Interaction

When people deliver a speech, they naturally move heads, and this rhythmic
head motion conveys prosodic information. However, generating a lip-synced
video while moving head naturally is challenging. While remarkably successful,
existing works either generate still talkingface videos or rely on
landmark/video frames as sparse/dense mapping guidance to generate head
movements, which leads to unrealistic or uncontrollable video synthesis. To
overcome the limitations, we propose a 3D-aware generative network along with a
hybrid embedding module and a non-linear composition module. Through modeling
the head motion and facial expressions1 explicitly, manipulating 3D animation
carefully, and embedding reference images dynamically, our approach achieves
controllable, photo-realistic, and temporally coherent talking-head videos with
natural head movements. Thoughtful experiments on several standard benchmarks
demonstrate that our method achieves significantly better results than the
state-of-the-art methods in both quantitative and qualitative comparisons. The
code is available on this https URL
lelechen63/Talking-head-Generation-with-Rhythmic-Head-Motion.

本文提出一种基于 3D-aware 生成网络、混合嵌入和非线性合成模块的方法，通过显式建模头部运动和面部表情，精心处理 3D 动画以及动态嵌入参考图像，实现了可控、逼真、时序连贯的说话者头像视频，并在多个标准基准测试中表现出优异的结果。