Current image captioning methods are usually trained via (penalized) maximum
likelihood estimation. However, the log-likelihood score of a caption does not
correlate well with human assessments of quality. Standard syntactic evaluation
metrics, such as BLEU, METEOR and ROUGE, are also not well correlated. The
newer SPICE and CIDEr metrics are better correlated, but have traditionally
been hard to optimize for. In this paper, we show how to use a policy gradient
(PG) method to directly optimize a linear combination of SPICE and CIDEr (a
combination we call SPIDEr): the SPICE score ensures our captions are
semantically faithful to the image, while CIDEr score ensures our captions are
syntactically fluent. The PG method we propose improves on the prior MIXER
approach, by using Monte Carlo rollouts instead of mixing MLE training with PG.
We show empirically that our algorithm leads to easier optimization and
improved results compared to MIXER. Finally, we show that using our PG method
we can optimize any of the metrics, including the proposed SPIDEr metric which
results in image captions that are strongly preferred by human raters compared
to captions generated by the same model but trained to optimize MLE or the COCO
metrics.

本文提出了一种基于策略梯度方法的优化算法，用于改进图像标题生成质量评估的 SPICE 和 CIDEr 指标的线性组合 SPIDEr，实验表明该算法优于之前的 MIXER 算法，并且使得使用 SPIDEr 指标训练的生成的图像标题比使用 MLE 或 COCO 指标训练的更受人类评估者欢迎。