Video generative models are receiving particular attention given their
ability to generate realistic and imaginative frames. Besides, these models are
also observed to exhibit strong 3D consistency, significantly enhancing their
potential to act as world simulators. In this work, we present Vidu4D, a novel
reconstruction model that excels in accurately reconstructing 4D (i.e.,
sequential 3D) representations from single generated videos, addressing
challenges associated with non-rigidity and frame distortion. This capability
is pivotal for creating high-fidelity virtual contents that maintain both
spatial and temporal coherence. At the core of Vidu4D is our proposed Dynamic
Gaussian Surfels (DGS) technique. DGS optimizes time-varying warping functions
to transform Gaussian surfels (surface elements) from a static state to a
dynamically warped state. This transformation enables a precise depiction of
motion and deformation over time. To preserve the structural integrity of
surface-aligned Gaussian surfels, we design the warped-state geometric
regularization based on continuous warping fields for estimating normals.
Additionally, we learn refinements on rotation and scaling parameters of
Gaussian surfels, which greatly alleviates texture flickering during the
warping process and enhances the capture of fine-grained appearance details.
Vidu4D also contains a novel initialization state that provides a proper start
for the warping fields in DGS. Equipping Vidu4D with an existing video
generative model, the overall framework demonstrates high-fidelity text-to-4D
generation in both appearance and geometry.

本论文介绍了一种名为 Vidu4D 的重建模型，通过使用动态高斯 Surfels 技术实现从单个生成的视频中准确重建 4D 表示，解决了非刚性运动和帧畸变等挑战，从而创建具有空间和时间一致性的高保真虚拟内容，并在外观和几何上展示了高保真的文本至 4D 生成。

Vidu4D：使用动态高斯黎曼表面对单一生成视频进行高保真度 4D 重建

Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with  Dynamic Gaussian Surfels

Video Anomaly Detection (VAD) is an open-set recognition task, which is
usually formulated as a one-class classification (OCC) problem, where training
data is comprised of videos with normal instances while test data contains both
normal and anomalous instances. Recent works have investigated the creation of
pseudo-anomalies (PAs) using only the normal data and making strong assumptions
about real-world anomalies with regards to abnormality of objects and speed of
motion to inject prior information about anomalies in an autoencoder (AE) based
reconstruction model during training. This work proposes a novel method for
generating generic spatio-temporal PAs by inpainting a masked out region of an
image using a pre-trained Latent Diffusion Model and further perturbing the
optical flow using mixup to emulate spatio-temporal distortions in the data. In
addition, we present a simple unified framework to detect real-world anomalies
under the OCC setting by learning three types of anomaly indicators, namely
reconstruction quality, temporal irregularity and semantic inconsistency.
Extensive experiments on four VAD benchmark datasets namely Ped2, Avenue,
ShanghaiTech and UBnormal demonstrate that our method performs on par with
other existing state-of-the-art PAs generation and reconstruction based methods
under the OCC setting. Our analysis also examines the transferability and
generalisation of PAs across these datasets, offering valuable insights by
identifying real-world anomalies through PAs.

通过使用预先训练的潜在扩散模型修复被屏蔽区域图像，并进一步通过混合来扰动光流以产生时间空间扭曲，提出了一种新的生成通用时空伪异常的方法。此外，通过学习重建质量、时间不规则性和语义不一致性等三种异常指标的简单统一框架，在开放集识别任务下检测现实世界的异常。在 Ped2、Avenue、ShanghaiTech 和 UBnormal 四个 VAD 基准数据集上进行了大量实验，证明我们的方法在 OCC 设置下的性能与其他现有的基于伪异常生成和重建的方法相当。同时我们的分析还检验了伪异常在这些数据集之间的可转移性和普适性，通过识别伪异常从而发现现实世界的异常提供了有价值的见解。