Visual text rendering poses a fundamental challenge for contemporary text-to-image generation models, with the core problem lying in text encoder deficiencies. To achieve accurate text rendering, we identify two crucial requirements for text encoders: character awareness and alignment with glyphs. Our solution involves crafting a series of customized text encoder, Glyph-ByT5, by fine-tuning the character-aware ByT5 encoder using a meticulously curated paired glyph-text dataset. We present an effective method for integrating Glyph-ByT5 with SDXL, resulting in the creation of the Glyph-SDXL model for design image generation. This significantly enhances text rendering accuracy, improving it from less than $20\%$ to nearly $90\%$ on our design image benchmark. Noteworthy is Glyph-SDXL's newfound ability for text paragraph rendering, achieving high spelling accuracy for tens to hundreds of characters with automated multi-line layouts. Finally, through fine-tuning Glyph-SDXL with a small set of high-quality, photorealistic images featuring visual text, we showcase a substantial improvement in scene text rendering capabilities in open-domain real images. These compelling outcomes aim to encourage further exploration in designing customized text encoders for diverse and challenging tasks.

通过定制化文本编码器Glyph-ByT5与SDXL的融合，我们提出了一种用于设计图像生成的Glyph-SDXL模型，大幅提高了文本渲染的准确性，并在打破原有的设计图像基准上，将渲染准确度从不到20%提高到近90%，同时还能实现自动多行布局，并通过细调与视觉文本相关的高质量照片，显著提升了开放域真实图像中场景文本渲染能力，为设计各种困难任务的定制化文本编码器的进一步探索提供了有力的倡议。

Glyph-ByT5: 准确视觉文本渲染的定制文本编码器