Video generation remains a challenging task due to spatiotemporal complexity
and the requirement of synthesizing diverse motions with temporal consistency.
Previous works attempt to generate videos in arbitrary lengths either in an
autoregressive manner or regarding time as a continuous signal. However, they
struggle to synthesize detailed and diverse motions with temporal coherence and
tend to generate repetitive scenes after a few time steps. In this work, we
argue that a single time-agnostic latent vector of style-based generator is
insufficient to model various and temporally-consistent motions. Hence, we
introduce additional time-dependent motion styles to model diverse motion
patterns. In addition, a Motion Style Attention modulation mechanism, dubbed as
MoStAtt, is proposed to augment frames with vivid dynamics for each specific
scale (i.e., layer), which assigns attention score for each motion style w.r.t
deconvolution filter weights in the target synthesis layer and softly attends
different motion styles for weight modulation. Experimental results show our
model achieves state-of-the-art performance on four unconditional $256^2$ video
synthesis benchmarks trained with only 3 frames per clip and produces better
qualitative results with respect to dynamic motions. Code and videos have been
made available at this https URL

通过引入一种 MoStAtt 机制，该研究提出了一种基于运动风格的视频生成模型，以增强帧的生动动态效果，并在四个条件下进行了使用大量跨帧的 256 × 256 合成视频，表现出了最先进的性能。

MoStGAN-V: 带有时间运动风格的视频生成

MoStGAN-V: Video Generation with Temporal Motion Styles

A large body of recent work targets semantically conditioned image
generation. Most such methods focus on the narrower task of pose transfer and
ignore the more challenging task of subject transfer that consists in not only
transferring the pose but also the appearance and background. In this work, we
introduce SCAM (Semantic Cross Attention Modulation), a system that encodes
rich and diverse information in each semantic region of the image (including
foreground and background), thus achieving precise generation with emphasis on
fine details. This is enabled by the Semantic Attention Transformer Encoder
that extracts multiple latent vectors for each semantic region, and the
corresponding generator that exploits these multiple latents by using semantic
cross attention modulation. It is trained only using a reconstruction setup,
while subject transfer is performed at test time. Our analysis shows that our
proposed architecture is successful at encoding the diversity of appearance in
each semantic region. Extensive experiments on the iDesigner and CelebAMask-HD
datasets show that SCAM outperforms SEAN and SPADE; moreover, it sets the new
state of the art on subject transfer.

本文提出了一个名为 SCAM (Semantic Cross Attention Modulation) 的系统，通过语义交叉注意力调节实现了精准图像生成，该系统可用于主体迁移的更具挑战性的任务，并在 iDesigner 和 CelebAMask-HD 数据集上超越了现有的 SEAN 和 SPADE 方法。

SCAM！基于语义交叉注意力调节的图像中人物迁移

SCAM! Transferring humans between images with Semantic Cross Attention Modulation

Despite considerable advancements with deep neural language models (LMs),
neural text generation still suffers from degeneration: the generated text is
repetitive, generic, self-contradictory, and often lacks commonsense. Our
analyses on sentence-level attention patterns in LMs reveal that neural
degeneration may be associated with insufficient learning of task-specific
characteristics by the attention mechanism. This finding motivates on-the-fly
attention modulation -- a simple but effective method that enables the
injection of priors into attention computation during inference. Automatic and
human evaluation results on three text generation benchmarks demonstrate that
attention modulation helps LMs generate text with enhanced fluency, creativity,
and commonsense reasoning, in addition to significantly reduce sentence-level
repetition.

本研究通过分析语言模型在句子级别的注意力模式，发现神经退化可能与注意机制对任务特征的学习不足有关。因此，提出了一种称为‘实时注意力调节’的方法，该方法在推理过程中向注意力计算注入学习先验知识，有效地改善了语言模型生成的文本的流畅性、创造性和常识推理能力，并显著减少了句子级别的重复。