In this work, we introduce a novel approach for creating controllable
dynamics in 3D-generated Gaussians using casually captured reference videos.
Our method transfers the motion of objects from reference videos to a variety
of generated 3D Gaussians across different categories, ensuring precise and
customizable motion transfer. We achieve this by employing blend skinning-based
non-parametric shape reconstruction to extract the shape and motion of
reference objects. This process involves segmenting the reference objects into
motion-related parts based on skinning weights and establishing shape
correspondences with generated target shapes. To address shape and temporal
inconsistencies prevalent in existing methods, we integrate physical
simulation, driving the target shapes with matched motion. This integration is
optimized through a displacement loss to ensure reliable and genuine dynamics.
Our approach supports diverse reference inputs, including humans, quadrupeds,
and articulated objects, and can generate dynamics of arbitrary length,
providing enhanced fidelity and applicability. Unlike methods heavily reliant
on diffusion video generation models, our technique offers specific and
high-quality motion transfer, maintaining both shape integrity and temporal
consistency.

通过从参考视频中提取对象形状和运动的新型非参数形状重建方法，结合物理模拟和匹配运动进行驱动，实现将参考视频中的对象运动转移到各种不同类别的生成的 3D 高斯函数，确保精确且可定制的运动转换。