The availability of large-scale multimodal datasets and advancements in
diffusion models have significantly accelerated progress in 4D content
generation. Most prior approaches rely on multiple image or video diffusion
models, utilizing score distillation sampling for optimization or generating
pseudo novel views for direct supervision. However, these methods are hindered
by slow optimization speeds and multi-view inconsistency issues. Spatial and
temporal consistency in 4D geometry has been extensively explored respectively
in 3D-aware diffusion models and traditional monocular video diffusion models.
Building on this foundation, we propose a strategy to migrate the temporal
consistency in video diffusion models to the spatial-temporal consistency
required for 4D generation. Specifically, we present a novel framework,
\textbf{Diffusion4D}, for efficient and scalable 4D content generation.
Leveraging a meticulously curated dynamic 3D dataset, we develop a 4D-aware
video diffusion model capable of synthesizing orbital views of dynamic 3D
assets. To control the dynamic strength of these assets, we introduce a
3D-to-4D motion magnitude metric as guidance. Additionally, we propose a novel
motion magnitude reconstruction loss and 3D-aware classifier-free guidance to
refine the learning and generation of motion dynamics. After obtaining orbital
views of the 4D asset, we perform explicit 4D construction with Gaussian
splatting in a coarse-to-fine manner. The synthesized multi-view consistent 4D
image set enables us to swiftly generate high-fidelity and diverse 4D assets
within just several minutes. Extensive experiments demonstrate that our method
surpasses prior state-of-the-art techniques in terms of generation efficiency
and 4D geometry consistency across various prompt modalities.

基于大规模多模态数据集和扩散模型的进展，我们提出了一种用于高效和可扩展的 4D 内容生成的新框架 Diffusion4D，具备时空一致性、多视角一致性和生成效率方面的优势。

Diffusion4D：基于视频扩散模型的快速时空一致的 4D 生成

Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video  Diffusion Models

Recent advancements in diffusion models for 2D and 3D content creation have
sparked a surge of interest in generating 4D content. However, the scarcity of
3D scene datasets constrains current methodologies to primarily object-centric
generation. To overcome this limitation, we present Comp4D, a novel framework
for Compositional 4D Generation. Unlike conventional methods that generate a
singular 4D representation of the entire scene, Comp4D innovatively constructs
each 4D object within the scene separately. Utilizing Large Language Models
(LLMs), the framework begins by decomposing an input text prompt into distinct
entities and maps out their trajectories. It then constructs the compositional
4D scene by accurately positioning these objects along their designated paths.
To refine the scene, our method employs a compositional score distillation
technique guided by the pre-defined trajectories, utilizing pre-trained
diffusion models across text-to-image, text-to-video, and text-to-3D domains.
Extensive experiments demonstrate our outstanding 4D content creation
capability compared to prior arts, showcasing superior visual quality, motion
fidelity, and enhanced object interactions.

使用大型语言模型 (LLMs) 以及预训练扩散模型，Comp4D 框架通过将文本提示分解为不同的实体并映射它们的轨迹，分别构建场景中的每个对象的四维 (4D) 表示，以获得卓越的 4D 内容生成能力，展现了更好的视觉质量、动作保真度和增强的对象交互。

Comp4D: 基于 LLM 的组合 4D 场景生成

Comp4D: LLM-Guided Compositional 4D Scene Generation

Remarkable progress has been made in 4D content generation recently. However,
existing methods suffer from long optimization time, lack of motion
controllability, and a low level of detail. In this paper, we introduce
DreamGaussian4D, an efficient 4D generation framework that builds on 4D
Gaussian Splatting representation. Our key insight is that the explicit
modeling of spatial transformations in Gaussian Splatting makes it more
suitable for the 4D generation setting compared with implicit representations.
DreamGaussian4D reduces the optimization time from several hours to just a few
minutes, allows flexible control of the generated 3D motion, and produces
animated meshes that can be efficiently rendered in 3D engines.

DreamGaussian4D 是一种高效的 4D 生成框架，它建立在 4D 高斯喷洒表示的基础上，通过显式地建模高斯喷洒中的空间变换，提高了 4D 生成的适用性，同时减少了优化时间，增加了对生成的三维动画的灵活控制，产生的动画网格在 3D 引擎中能被高效渲染。