We present Hunyuan-DiT, a text-to-image diffusion transformer with
fine-grained understanding of both English and Chinese. To construct
Hunyuan-DiT, we carefully design the transformer structure, text encoder, and
positional encoding. We also build from scratch a whole data pipeline to update
and evaluate data for iterative model optimization. For fine-grained language
understanding, we train a Multimodal Large Language Model to refine the
captions of the images. Finally, Hunyuan-DiT can perform multi-turn multimodal
dialogue with users, generating and refining images according to the context.
Through our holistic human evaluation protocol with more than 50 professional
human evaluators, Hunyuan-DiT sets a new state-of-the-art in Chinese-to-image
generation compared with other open-source models. Code and pretrained models
are publicly available at github.com/Tencent/HunyuanDiT

提出了混元 - DiT，一种具有对英文和汉语进行细粒度理解能力的文本到图像扩散变压器，并基于其精心设计的变压器结构、文本编码器和位置编码，构建了全新的数据流水线，以更新和评估迭代模型优化的数据。通过训练一个多模态大型语言模型来完善图像的标题，最终，混元 - DiT 可以与用户进行多轮多模态对话，根据上下文生成和完善图像。通过与其他开源模型进行综合人类评估，混元 - DiT 在中文到图像生成方面创造了新的最先进水平。代码和预训练模型可在 github.com/Tencent/HunyuanDiT 公开获取。