Current text recognition systems, including those for handwritten scripts and
scene text, have relied heavily on image synthesis and augmentation, since it
is difficult to realize real-world complexity and diversity through collecting
and annotating enough real text images. In this paper, we explore the problem
of text image generation, by taking advantage of the powerful abilities of
Diffusion Models in generating photo-realistic and diverse image samples with
given conditions, and propose a method called Conditional Text Image Generation
with Diffusion Models (CTIG-DM for short). To conform to the characteristics of
text images, we devise three conditions: image condition, text condition, and
style condition, which can be used to control the attributes, contents, and
styles of the samples in the image generation process. Specifically, four text
image generation modes, namely: (1) synthesis mode, (2) augmentation mode, (3)
recovery mode, and (4) imitation mode, can be derived by combining and
configuring these three conditions. Extensive experiments on both handwritten
and scene text demonstrate that the proposed CTIG-DM is able to produce image
samples that simulate real-world complexity and diversity, and thus can boost
the performance of existing text recognizers. Besides, CTIG-DM shows its
appealing potential in domain adaptation and generating images containing
Out-Of-Vocabulary (OOV) words.

本文提出一种名为 CTIG-DM 的新方法，它结合了图像条件、文本条件和风格条件来控制样本在图像生成过程中的属性、内容和风格。经过大量的实验，证明 CTIG-DM 能够产生模拟现实世界复杂性和多样性的图像样本，具有很强的潜力用于领域适应和生成包含 OOV（Out-Of-Vocabulary）词的图像。