We present a novel approach for generating high-quality, spatio-temporally
coherent human videos from a single image under arbitrary viewpoints. Our
framework combines the strengths of U-Nets for accurate condition injection and
diffusion transformers for capturing global correlations across viewpoints and
time. The core is a cascaded 4D transformer architecture that factorizes
attention across views, time, and spatial dimensions, enabling efficient
modeling of the 4D space. Precise conditioning is achieved by injecting human
identity, camera parameters, and temporal signals into the respective
transformers. To train this model, we curate a multi-dimensional dataset
spanning images, videos, multi-view data and 3D/4D scans, along with a
multi-dimensional training strategy. Our approach overcomes the limitations of
previous methods based on GAN or UNet-based diffusion models, which struggle
with complex motions and viewpoint changes. Through extensive experiments, we
demonstrate our method's ability to synthesize realistic, coherent and
free-view human videos, paving the way for advanced multimedia applications in
areas such as virtual reality and animation. Our project website is
this https URL

我们提出了一种新颖的方法，可以从单个图像在任意视角下生成高质量、时空连贯的人类视频。我们的框架结合了 U-Nets 的精确条件注入和扩散变换器的在视角和时间上捕捉全局相关性的优势。核心是一个级联的 4D 变换器架构，可以分解关注点以跨视角、时间和空间维度，实现对 4D 空间的高效建模。我们通过将人类身份、相机参数和时间信号注入到相应的变换器中来实现精确的条件设定。为了训练这个模型，我们策划了一个跨图像、视频、多视角数据和 3D/4D 扫描的多维数据集以及多维训练策略。我们的方法克服了以 GAN 或基于 UNet 的扩散模型为基础的先前方法在处理复杂动作和视角变化时的局限性。通过广泛的实验证明，我们的方法能够合成逼真、连贯和自由观察的人类视频，为虚拟现实和动画等领域的先进多媒体应用铺平了道路。我们的项目网站为 https URL。