Scene text synthesis involves rendering specified texts onto arbitrary
images. Current methods typically formulate this task in an end-to-end manner
but lack effective character-level guidance during training. Besides, their
text encoders, pre-trained on a single font type, struggle to adapt to the
diverse font styles encountered in practical applications. Consequently, these
methods suffer from character distortion, repetition, and absence, particularly
in polystylistic scenarios. To this end, this paper proposes DreamText for
high-fidelity scene text synthesis. Our key idea is to reconstruct the
diffusion training process, introducing more refined guidance tailored to this
task, to expose and rectify the model's attention at the character level and
strengthen its learning of text regions. This transformation poses a hybrid
optimization challenge, involving both discrete and continuous variables. To
effectively tackle this challenge, we employ a heuristic alternate optimization
strategy. Meanwhile, we jointly train the text encoder and generator to
comprehensively learn and utilize the diverse font present in the training
dataset. This joint training is seamlessly integrated into the alternate
optimization process, fostering a synergistic relationship between learning
character embedding and re-estimating character attention. Specifically, in
each step, we first encode potential character-generated position information
from cross-attention maps into latent character masks. These masks are then
utilized to update the representation of specific characters in the current
step, which, in turn, enables the generator to correct the character's
attention in the subsequent steps. Both qualitative and quantitative results
demonstrate the superiority of our method to the state of the art.

本研究提出了 DreamText 方法，旨在通过增加细化的字符级指导以及多样的字体训练，提高高保真场景文本合成效果，并通过融合离散和连续变量的混合优化策略来解决该优化问题。实验结果表明，本方法在质量和数量上优于现有方法。

高保真场景文本合成

High Fidelity Scene Text Synthesis

There are various font styles in the world. Different styles give different
impressions and readability. This paper analyzes the relationship between font
styles and contextual factors that might affect font style selection with
large-scale datasets. For example, we will analyze the relationship between
font style and its surrounding object (such as ``bus'') by using about 800,000
words in the Open Images dataset. We also use a book cover dataset to analyze
the relationship between font styles with book genres. Moreover, the meaning of
the word is assumed as another contextual factor. For these numeric analyses,
we utilize our own font-style feature extraction model and word2vec. As a
result of co-occurrence-based relationship analysis, we found several instances
of specific font styles being used for specific contextual factors.

研究探讨了世界上不同的字体风格和它们对阅读体验和印象的影响，同时借助大规模数据集，分析字体风格的选择与上下文因素，如周围对象和单词意义的关系，从而发现了特定字体风格在特定上下文因素下的使用实例。