We address the task of open-world class-agnostic object detection, i.e.,
detecting every object in an image by learning from a limited number of base
object classes. State-of-the-art RGB-based models suffer from overfitting the
training classes and often fail at detecting novel-looking objects. This is
because RGB-based models primarily rely on appearance similarity to detect
novel objects and are also prone to overfitting short-cut cues such as textures
and discriminative parts. To address these shortcomings of RGB-based object
detectors, we propose incorporating geometric cues such as depth and normals,
predicted by general-purpose monocular estimators. Specifically, we use the
geometric cues to train an object proposal network for pseudo-labeling
unannotated novel objects in the training set. Our resulting Geometry-guided
Open-world Object Detector (GOOD) significantly improves detection recall for
novel object categories and already performs well with only a few training
classes. Using a single "person" class for training on the COCO dataset, GOOD
surpasses SOTA methods by 5.0% AR@100, a relative improvement of 24%.

本文提出了一种基于几何线索的开放式物体检测方法 GOOD（Geometry-guided Open-world Object Detector），使用深度和法线等几何线索来伪标记未标记的新物体，从而显著提高新物体类别的检测召回率，通过在 COCO 数据集上使用单一的 “人” 类别进行训练，GOOD 在 AR@100 上相对于 SOTA 方法提高了 24％，达到了 5.0％的改进。

探索几何线索以检测开放世界中的物体

GOOD: Exploring Geometric Cues for Detecting Objects in an Open World

Image captioning is an important task for benchmarking visual reasoning and
for enabling accessibility for people with vision impairments. However, as in
many machine learning settings, social biases can influence image captioning in
undesirable ways. In this work, we study bias propagation pathways within image
captioning, focusing specifically on the COCO dataset. Prior work has analyzed
gender bias in captions using automatically-derived gender labels; here we
examine racial and intersectional biases using manual annotations. Our first
contribution is in annotating the perceived gender and skin color of 28,315 of
the depicted people after obtaining IRB approval. Using these annotations, we
compare racial biases present in both manual and automatically-generated image
captions. We demonstrate differences in caption performance, sentiment, and
word choice between images of lighter versus darker-skinned people. Further, we
find the magnitude of these differences to be greater in modern captioning
systems compared to older ones, thus leading to concerns that without proper
consideration and mitigation these differences will only become increasingly
prevalent. Code and data is available at
this https URL .

本研究分析了图像描述中的社会偏见以及种族偏见，具体关注于 COCO 数据集中的感知性别和肤色标注等因素。结果表明图像描述模型对于不同肤色的人物的表述、情感和词汇选择存在差异，并发现这种差异在现代模型中更加明显。