Text-to-Image (T2I) Synthesis has made tremendous strides in enhancing
synthesized image quality, but current datasets evaluate model performance only
on descriptive, instruction-based prompts. Real-world news image captions take
a more pragmatic approach, providing high-level situational and Named-Entity
(NE) information and limited physical object descriptions, making them
abstractive. To evaluate the ability of T2I models to capture intended subjects
from news captions, we introduce the Abstractive News Captions with High-level
cOntext Representation (ANCHOR) dataset, containing 70K+ samples sourced from 5
different news media organizations. With Large Language Models (LLM) achieving
success in language and commonsense reasoning tasks, we explore the ability of
different LLMs to identify and understand key subjects from abstractive
captions. Our proposed method Subject-Aware Finetuning (SAFE), selects and
enhances the representation of key subjects in synthesized images by leveraging
LLM-generated subject weights. It also adapts to the domain distribution of
news images and captions through custom Domain Fine-tuning, outperforming
current T2I baselines on ANCHOR. By launching the ANCHOR dataset, we hope to
motivate research in furthering the Natural Language Understanding (NLU)
capabilities of T2I models.

介绍了一种针对新闻标题的抽象性抽象画面生成数据集，通过使用大型语言模型，探索了不同语言模型在抽象标题中识别和理解关键主题的能力，并提出了一种基于主题感知微调和领域微调的方法以改进生成图像质量。旨在促进进一步发展文本到图像合成模型的自然语言理解能力的研究。