We introduce a method to generate temporally coherent human animation from a
single image, a video, or a random noise. This problem has been formulated as
modeling of an auto-regressive generation, i.e., to regress past frames to
decode future frames. However, such unidirectional generation is highly prone
to motion drifting over time, generating unrealistic human animation with
significant artifacts such as appearance distortion. We claim that
bidirectional temporal modeling enforces temporal coherence on a generative
network by largely suppressing the motion ambiguity of human appearance. To
prove our claim, we design a novel human animation framework using a denoising
diffusion model: a neural network learns to generate the image of a person by
denoising temporal Gaussian noises whose intermediate results are
cross-conditioned bidirectionally between consecutive frames. In the
experiments, our method demonstrates strong performance compared to existing
unidirectional approaches with realistic temporal coherence

本文提出了一种通过使用双向时间建模的去噪扩散模型生成与真实人类动画高度相似的方法。

双向时间扩散模型 —— 用于保持时间连续的人物动画

Bidirectional Temporal Diffusion Model for Temporally Consistent Human  Animation

Large language models (LLMs) based on the generative pre-training transformer
(GPT) have demonstrated remarkable effectiveness across a diverse range of
downstream tasks. Inspired by the advancements of the GPT, we present PointGPT,
a novel approach that extends the concept of GPT to point clouds, addressing
the challenges associated with disorder properties, low information density,
and task gaps. Specifically, a point cloud auto-regressive generation task is
proposed to pre-train transformer models. Our method partitions the input point
cloud into multiple point patches and arranges them in an ordered sequence
based on their spatial proximity. Then, an extractor-generator based
transformer decoder, with a dual masking strategy, learns latent
representations conditioned on the preceding point patches, aiming to predict
the next one in an auto-regressive manner. Our scalable approach allows for
learning high-capacity models that generalize well, achieving state-of-the-art
performance on various downstream tasks. In particular, our approach achieves
classification accuracies of 94.9% on the ModelNet40 dataset and 93.4% on the
ScanObjectNN dataset, outperforming all other transformer models. Furthermore,
our method also attains new state-of-the-art accuracies on all four few-shot
learning benchmarks.

提出了 PointGPT 方法来扩展 GPT 模型到点云中进行自回归生成任务的预训练，其通过多个点序列来实现学习潜在的表示，并在各种下游任务中实现了最先进的性能，特别是在 few-shot 学习方面也产生了新的 state-of-the-art 结果。