Generating stylized captions for an image is an emerging topic in image captioning. Given an image as input, it requires the system to generate a caption that has a specific style (e.g., humorous, romantic, positive, and negative) while describing the image content semantically accurately. In this paper, we propose a novel stylized image captioning model that effectively takes both requirements into consideration. To this end, we first devise a new variant of LSTM, named style-factual LSTM, as the building block of our model. It uses two groups of matrices to capture the factual and stylized knowledge, respectively, and automatically learns the word-level weights of the two groups based on previous context. In addition, when we train the model to capture stylized elements, we propose an adaptive learning approach based on a reference factual model, it provides factual knowledge to the model as the model learns from stylized caption labels, and can adaptively compute how much information to supply at each time step. We evaluate our model on two stylized image captioning datasets, which contain humorous/romantic captions and positive/negative captions, respectively. Experiments shows that our proposed model outperforms the state-of-the-art approaches, without using extra ground truth supervision.

本文提出了一种新颖的基于style-factual LSTM和自适应学习的图像标注模型，能够同时考虑图像的内容和特定的风格要求，如幽默，浪漫，正面和负面。实验结果表明，该模型在两个不同的图像标注数据集上（幽默/浪漫标注以及正面/负面标注）的表现均超过了目前的最先进方法，且不需要额外的监督信息。

基于自适应学习和注意力机制的风格化图像字幕生成：客观描述或情感表现？