One of the major challenges in training text-to-image generation models is
the need of a large number of high-quality image-text pairs. While image
samples are often easily accessible, the associated text descriptions typically
require careful human captioning, which is particularly time- and
cost-consuming. In this paper, we propose the first work to train text-to-image
generation models without any text data. Our method leverages the well-aligned
multi-modal semantic space of the powerful pre-trained CLIP model: the
requirement of text-conditioning is seamlessly alleviated via generating text
features from image features. Extensive experiments are conducted to illustrate
the effectiveness of the proposed method. We obtain state-of-the-art results in
the standard text-to-image generation tasks. Importantly, the proposed
language-free model outperforms most existing models trained with full
image-text pairs. Furthermore, our method can be applied in fine-tuning
pre-trained models, which saves both training time and cost in training
text-to-image generation models. Our pre-trained model obtains competitive
results in zero-shot text-to-image generation on the MS-COCO dataset, yet with
around only 1% of the model size and training data size relative to the
recently proposed large DALL-E model.

本文提出了一种基于 CLIP 模型的无语言数据训练文本到图像生成模型的方法，采用图像特征生成文本特征，实现了对文本数据的无缝融合。实验结果表明，该方法在标准文本到图像生成任务中取得了最好的表现，并且优于大多数使用完整图像 - 文本对进行训练的现有模型。此外，我们的方法可以应用于微调预训练模型，从而节省文本到图像生成模型的培训时间和成本。

LAFITE：面向文本图像生成的语言无关训练

LAFITE: Towards Language-Free Training for Text-to-Image Generation

In this research, we present a heuristic method for character recognition.
For this purpose, a sketch is constructed from the image that contains the
character to be recognized. This sketch contains the most important pixels of
image that are representatives of original image. These points are the most
probable points in pixel-by-pixel matching of image that adapt to target image.
Furthermore, a technique called gravity shifting is utilized for taking over
the problem of elongation of characters. The consequence of combining sketch
and gravity techniques leaded to a language free character recognition method.
This method can be implemented independently for real-time uses or in
combination of other classifiers as a feature extraction algorithm. Low
complexity and acceptable performance are the most impressive features of this
method that let it to be simply implemented in mobile and battery-limited
computing devices. Results show that in the best case 86% of accuracy is
obtained and in the worst case 28% of recognized characters are accurate.

本研究介绍了一种基于启发式方法的字符识别方法，利用草图和重力转移技术实现无语言限制的低复杂度字符识别，并可独立运用于实时目标检测中。在测试中，最高可达 86% 的识别率，最低为 28%。