Reconstructing 3D human bodies from realistic motion sequences remains a
challenge due to pervasive and complex occlusions. Current methods struggle to
capture the dynamics of occluded body parts, leading to model penetration and
distorted motion. RemoCap leverages Spatial Disentanglement (SD) and Motion
Disentanglement (MD) to overcome these limitations. SD addresses occlusion
interference between the target human body and surrounding objects. It achieves
this by disentangling target features along the dimension axis. By aligning
features based on their spatial positions in each dimension, SD isolates the
target object's response within a global window, enabling accurate capture
despite occlusions. The MD module employs a channel-wise temporal shuffling
strategy to simulate diverse scene dynamics. This process effectively
disentangles motion features, allowing RemoCap to reconstruct occluded parts
with greater fidelity. Furthermore, this paper introduces a sequence velocity
loss that promotes temporal coherence. This loss constrains inter-frame
velocity errors, ensuring the predicted motion exhibits realistic consistency.
Extensive comparisons with state-of-the-art (SOTA) methods on benchmark
datasets demonstrate RemoCap's superior performance in 3D human body
reconstruction. On the 3DPW dataset, RemoCap surpasses all competitors,
achieving the best results in MPVPE (81.9), MPJPE (72.7), and PA-MPJPE (44.1)
metrics. Codes are available at this https URL

RemoCap 使用空间解缠和动作解缠方法以解决复杂遮挡对 3D 人体重建造成的困难，实现溢出物体区域的准确捕获并重建遮挡的部分。

RemoCap: 动作捕捉的解缠表示学习

RemoCap: Disentangled Representation Learning for Motion Capture

The essence of a video lies in its dynamic motions, including character
actions, object movements, and camera movements. While text-to-video generative
diffusion models have recently advanced in creating diverse contents,
controlling specific motions through text prompts remains a significant
challenge. A primary issue is the coupling of appearance and motion, often
leading to overfitting on appearance. To tackle this challenge, we introduce
MotionCrafter, a novel one-shot instance-guided motion customization method.
MotionCrafter employs a parallel spatial-temporal architecture that injects the
reference motion into the temporal component of the base model, while the
spatial module is independently adjusted for character or style control. To
enhance the disentanglement of motion and appearance, we propose an innovative
dual-branch motion disentanglement approach, comprising a motion
disentanglement loss and an appearance prior enhancement strategy. During
training, a frozen base model provides appearance normalization, effectively
separating appearance from motion and thereby preserving diversity.
Comprehensive quantitative and qualitative experiments, along with user
preference tests, demonstrate that MotionCrafter can successfully integrate
dynamic motions while preserving the coherence and quality of the base model
with a wide range of appearance generation capabilities. Codes are available at
this https URL

MotionCrafter 是一种新型一次性实例引导的动作定制方法，通过并行的时空架构注入参考动作到基础模型的时间部分，同时独立调整空间模块以进行角色或风格控制，以增强动作与外观的解耦，进而维持多样性，量化和定性实验证明了 MotionCrafter 在整合动态动作、保持一致性和提供广泛外观生成能力方面的成功。

MotionCrafter：一次性定制扩散模型的运动

MotionCrafter: One-Shot Motion Customization of Diffusion Models

Talking head generation is to generate video based on a given source identity
and target motion. However, current methods face several challenges that limit
the quality and controllability of the generated videos. First, the generated
face often has unexpected deformation and severe distortions. Second, the
driving image does not explicitly disentangle movement-relevant information,
such as poses and expressions, which restricts the manipulation of different
attributes during generation. Third, the generated videos tend to have
flickering artifacts due to the inconsistency of the extracted landmarks
between adjacent frames. In this paper, we propose a novel model that produces
high-fidelity talking head videos with free control over head pose and
expression. Our method leverages both self-supervised learned landmarks and 3D
face model-based landmarks to model the motion. We also introduce a novel
motion-aware multi-scale feature alignment module to effectively transfer the
motion without face distortion. Furthermore, we enhance the smoothness of the
synthesized talking head videos with a feature context adaptation and
propagation module. We evaluate our model on challenging datasets and
demonstrate its state-of-the-art performance. More information is available at
this https URL

本文提出了一种新模型，通过利用自监督学习技术和三维人脸模型中的标志点来对运动进行建模，并引入了新的运动感知多尺度特征对齐模块来进行视频合成，从而实现了对头部姿态和表情的自由控制，并且得到了最优质的合成音频视频输出。