The diffusion model has been proven a powerful generative model in recent
years, yet remains a challenge in generating visual text. Several methods
alleviated this issue by incorporating explicit text position and content as
guidance on where and what text to render. However, these methods still suffer
from several drawbacks, such as limited flexibility and automation, constrained
capability of layout prediction, and restricted style diversity. In this paper,
we present TextDiffuser-2, aiming to unleash the power of language models for
text rendering. Firstly, we fine-tune a large language model for layout
planning. The large language model is capable of automatically generating
keywords for text rendering and also supports layout modification through
chatting. Secondly, we utilize the language model within the diffusion model to
encode the position and texts at the line level. Unlike previous methods that
employed tight character-level guidance, this approach generates more diverse
text images. We conduct extensive experiments and incorporate user studies
involving human participants as well as GPT-4V, validating TextDiffuser-2's
capacity to achieve a more rational text layout and generation with enhanced
diversity. The code and model will be available at
https://aka.ms/textdiffuser-2.

通过使用语言模型进行布局规划，TextDiffuser-2 能够实现更合理的文本布局和具有增强多样性的生成。

TextDiffuser-2：释放语言模型在文本呈现中的能力

TextDiffuser-2: Unleashing the Power of Language Models for Text  Rendering

Styled online in-between motion generation has important application
scenarios in computer animation and games. Its core challenge lies in the need
to satisfy four critical requirements simultaneously: generation speed, motion
quality, style diversity, and synthesis controllability. While the first two
challenges demand a delicate balance between simple fast models and learning
capacity for generation quality, the latter two are rarely investigated
together in existing methods, which largely focus on either control without
style or uncontrolled stylized motions. To this end, we propose a Real-time
Stylized Motion Transition method (RSMT) to achieve all aforementioned goals.
Our method consists of two critical, independent components: a general motion
manifold model and a style motion sampler. The former acts as a high-quality
motion source and the latter synthesizes styled motions on the fly under
control signals. Since both components can be trained separately on different
datasets, our method provides great flexibility, requires less data, and
generalizes well when no/few samples are available for unseen styles. Through
exhaustive evaluation, our method proves to be fast, high-quality, versatile,
and controllable. The code and data are available at
{this https URL}

本文提出使用实时样式化运动转换方法 (RSMT) 在满足速度、质量、多样性和可控性的要求下，生成在线样式化运动，其核心在于提出了一个运动流形模型和样式运动采样器，可以分别对不同数据集进行单独训练和综合评估.