Despite weakly supervised object detection (WSOD) being a promising step
toward evading strong instance-level annotations, its capability is confined to
closed-set categories within a single training dataset. In this paper, we
propose a novel weakly supervised open-vocabulary object detection framework,
namely WSOVOD, to extend traditional WSOD to detect novel concepts and utilize
diverse datasets with only image-level annotations. To achieve this, we explore
three vital strategies, including dataset-level feature adaptation, image-level
salient object localization, and region-level vision-language alignment. First,
we perform data-aware feature extraction to produce an input-conditional
coefficient, which is leveraged into dataset attribute prototypes to identify
dataset bias and help achieve cross-dataset generalization. Second, a
customized location-oriented weakly supervised region proposal network is
proposed to utilize high-level semantic layouts from the category-agnostic
segment anything model to distinguish object boundaries. Lastly, we introduce a
proposal-concept synchronized multiple-instance network, i.e., object mining
and refinement with visual-semantic alignment, to discover objects matched to
the text embeddings of concepts. Extensive experiments on Pascal VOC and MS
COCO demonstrate that the proposed WSOVOD achieves new state-of-the-art
compared with previous WSOD methods in both close-set object localization and
detection tasks. Meanwhile, WSOVOD enables cross-dataset and open-vocabulary
learning to achieve on-par or even better performance than well-established
fully-supervised open-vocabulary object detection (FSOVOD).

提出了一种弱监督开放式词汇目标检测（WSOVOD）框架，扩展了传统的弱监督目标检测方法，实现了对新概念的检测和仅使用图像级别标注的多样数据集的利用。

弱监督开放词汇物体检测

Weakly Supervised Open-Vocabulary Object Detection

Automated detection of Gallbladder Cancer (GBC) from Ultrasound (US) images
is an important problem, which has drawn increased interest from researchers.
However, most of these works use difficult-to-acquire information such as
bounding box annotations or additional US videos. In this paper, we focus on
GBC detection using only image-level labels. Such annotation is usually
available based on the diagnostic report of a patient, and do not require
additional annotation effort from the physicians. However, our analysis reveals
that it is difficult to train a standard image classification model for GBC
detection. This is due to the low inter-class variance (a malignant region
usually occupies only a small portion of a US image), high intra-class variance
(due to the US sensor capturing a 2D slice of a 3D object leading to large
viewpoint variations), and low training data availability. We posit that even
when we have only the image level label, still formulating the problem as
object detection (with bounding box output) helps a deep neural network (DNN)
model focus on the relevant region of interest. Since no bounding box
annotations is available for training, we pose the problem as weakly supervised
object detection (WSOD). Motivated by the recent success of transformer models
in object detection, we train one such model, DETR, using
multi-instance-learning (MIL) with self-supervised instance selection to suit
the WSOD task. Our proposed method demonstrates an improvement of AP and
detection sensitivity over the SOTA transformer-based and CNN-based WSOD
methods. Project page is at this https URL

通过使用图像级标签进行 GBC 检测，本文提出了一种使用自我监督实例选择的多实例学习方法训练 DETR 模型的弱监督目标检测算法，相较于目前最先进的基于变换器和卷积神经网络的弱监督目标检测方法，我们的方法在 AP 和检测灵敏度上表现出较大的改进。