Generating a short story out of an image is arduous. Unlike image captioning,
story generation from an image poses multiple challenges: preserving the story
coherence, appropriately assessing the quality of the story, steering the
generated story into a certain style, and addressing the scarcity of
image-story pair reference datasets limiting supervision during training. In
this work, we introduce Plug-and-Play Story Teller (PPST) and improve
image-to-story generation by: 1) alleviating the data scarcity problem by
incorporating large pre-trained models, namely CLIP and GPT-2, to facilitate a
fluent image-to-text generation with minimal supervision, and 2) enabling a
more style-relevant generation by incorporating stylistic adapters to control
the story generation. We conduct image-to-story generation experiments with
non-styled, romance-styled, and action-styled PPST approaches and compare our
generated stories with those of previous work over three aspects, i.e., story
coherence, image-story relevance, and style fitness, using both automatic and
human evaluation. The results show that PPST improves story coherence and has
better image-story relevance, but has yet to be adequately stylistic.

本文通过加入预训练模型 CLIP 和 GPT-2 解决数据稀缺性问题，加入风格适配器控制故事生成，尝试解决从图像到故事的生成难题，实现了包括非风格化、浪漫和动作等多种风格的故事生成。结果表明，该方法改善了故事的连贯性和图像与故事的相关性，但风格仍有待改进。