Is aesthetic impact different from beauty? Is visual salience a reflection of
its capacity for effective communication? We present Impressions, a novel
dataset through which to investigate the semiotics of images, and how specific
visual features and design choices can elicit specific emotions, thoughts and
beliefs. We posit that the impactfulness of an image extends beyond formal
definitions of aesthetics, to its success as a communicative act, where style
contributes as much to meaning formation as the subject matter. However, prior
image captioning datasets are not designed to empower state-of-the-art
architectures to model potential human impressions or interpretations of
images. To fill this gap, we design an annotation task heavily inspired by
image analysis techniques in the Visual Arts to collect 1,440 image-caption
pairs and 4,320 unique annotations exploring impact, pragmatic image
description, impressions, and aesthetic design choices. We show that existing
multimodal image captioning and conditional generation models struggle to
simulate plausible human responses to images. However, this dataset
significantly improves their ability to model impressions and aesthetic
evaluations of images through fine-tuning and few-shot adaptation.

通过独特的图像语义学数据集 Impressions，研究图像的符号学以及特定的视觉特征和设计选择如何引发特定的情感、思考和信念，证明了对图像印象的影响力超出了形式美学的描述，而是涉及其作为传达行为的成功，其中风格为意义形成做出的贡献与主题内容一样重要。通过 Fine-tuning 和少样本适应，对现有的多模态图像字幕和条件生成模型进行训练，可以模拟出对图像的合理人类反应，从而有效地模拟图像的印象和美学评价。