A creative image-and-text generative AI system mimics humans' extraordinary
abilities to provide users with diverse and comprehensive caption suggestions,
as well as rich image creations. In this work, we demonstrate such an AI
creation system to produce both diverse captions and rich images. When users
imagine an image and associate it with multiple captions, our system paints a
rich image to reflect all captions faithfully. Likewise, when users upload an
image, our system depicts it with multiple diverse captions. We propose a
unified multi-modal framework to achieve this goal. Specifically, our framework
jointly models image-and-text representations with a Transformer network, which
supports rich image creation by accepting multiple captions as input. We
consider the relations among input captions to encourage diversity in training
and adopt a non-autoregressive decoding strategy to enable real-time inference.
Based on these, our system supports both diverse captions and rich images
generations. Our code is available online.

该研究采用创意图像和文本生成 AI 系统，通过多模态框架实现了多样化的标题和图像创建，同时采用 Transformer 网络模型支持多种标题参考，采用非自回归解码策略实现实时推断。