In this paper, we investigate the impact of objects on gender bias in image
captioning systems. Our results show that only gender-specific objects have a
strong gender bias (e.g., women-lipstick). In addition, we propose a visual
semantic-based gender score that measures the degree of bias and can be used as
a plug-in for any image captioning system. Our experiments demonstrate the
utility of the gender score, since we observe that our score can measure the
bias relation between a caption and its related gender; therefore, our score
can be used as an additional metric to the existing Object Gender Co-Occ
approach. Code and data are publicly available at
https://github.com/ahmedssabir/GenderScore.

研究了物体对图像字幕系统中的性别偏见的影响，结果表明只有特定于性别的物体才有强烈的性别偏见（例如，女性口红）。此外，提出了一种基于视觉语义的性别得分，用来度量偏见程度并可作为任何图像字幕系统的插件。实验证明了性别得分的实用性，因为我们观察到我们的得分可以测量字幕和相关性别之间的偏见关系；因此，我们的得分可以作为现有的对象性别共项方法的补充指标。代码和数据公开可用于 https://github.com/ahmedssabir/GenderScore。

口红女性：测量对象与其相关性别之间的偏见

Women Wearing Lipstick: Measuring the Bias Between an Object and Its  Related Gender

Popular metrics used for evaluating image captioning systems, such as BLEU
and CIDEr, provide a single score to gauge the system's overall effectiveness.
This score is often not informative enough to indicate what specific errors are
made by a given system. In this study, we present a fine-grained evaluation
method REO for automatically measuring the performance of image captioning
systems. REO assesses the quality of captions from three perspectives: 1)
Relevance to the ground truth, 2) Extraness of the content that is irrelevant
to the ground truth, and 3) Omission of the elements in the images and human
references. Experiments on three benchmark datasets demonstrate that our method
achieves a higher consistency with human judgments and provides more intuitive
evaluation results than alternative metrics.

本研究提出了细粒度评估方法 REO，从与实际参照的相关性、多余性和缺少性三个方面评估图像字幕系统的性能，实验证明与人类判断更具一致性，结果更直观。