Generating text-editable and pose-controllable character videos have an
imperious demand in creating various digital human. Nevertheless, this task has
been restricted by the absence of a comprehensive dataset featuring paired
video-pose captions and the generative prior models for videos. In this work,
we design a novel two-stage training scheme that can utilize easily obtained
datasets (i.e.,image pose pair and pose-free video) and the pre-trained
text-to-image (T2I) model to obtain the pose-controllable character videos.
Specifically, in the first stage, only the keypoint-image pairs are used only
for a controllable text-to-image generation. We learn a zero-initialized
convolu- tional encoder to encode the pose information. In the second stage, we
finetune the motion of the above network via a pose-free video dataset by
adding the learnable temporal self-attention and reformed cross-frame
self-attention blocks. Powered by our new designs, our method successfully
generates continuously pose-controllable character videos while keeps the
editing and concept composition ability of the pre-trained T2I model. The code
and models will be made publicly available.

本研究提出一种新的双阶段训练方法，可利用易获得的数据集和预训练的 text-to-image（T2I）模型，生成可编辑和姿态可控的字符视频。这种方法成功地生成了连续的姿态可控字符视频，同时保持了预训练 T2I 模型的编辑和概念构成能力。

跟随你的姿势：使用无姿态视频进行姿势引导的文本到视频生成

Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free  Videos

Unprecedented access to multi-temporal satellite imagery has opened new
perspectives for a variety of Earth observation tasks. Among them,
pixel-precise panoptic segmentation of agricultural parcels has major economic
and environmental implications. While researchers have explored this problem
for single images, we argue that the complex temporal patterns of crop
phenology are better addressed with temporal sequences of images. In this
paper, we present the first end-to-end, single-stage method for panoptic
segmentation of Satellite Image Time Series (SITS). This module can be combined
with our novel image sequence encoding network which relies on temporal
self-attention to extract rich and adaptive multi-scale spatio-temporal
features. We also introduce PASTIS, the first open-access SITS dataset with
panoptic annotations. We demonstrate the superiority of our encoder for
semantic segmentation against multiple competing architectures, and set up the
first state-of-the-art of panoptic segmentation of SITS. Our implementation and
PASTIS are publicly available.

本篇论文提出了一种全新的端到端的、适合于处理卫星图像时间序列的像素级全景分割方法，并通过引入时序的注意力机制，提取了丰富且自适应的多尺度时空特征，同时还开发了第一个完全公开的卫星图像时间序列数据集 PASTIS，并展示了该方法在语义分割任务上的卓越表现。