Object detection with transformers (DETR) reaches competitive performance
with Faster R-CNN via a transformer encoder-decoder architecture. Inspired by
the great success of pre-training transformers in natural language processing,
we propose a pretext task named random query patch detection to Unsupervisedly
Pre-train DETR (UP-DETR) for object detection. Specifically, we randomly crop
patches from the given image and then feed them as queries to the decoder. The
model is pre-trained to detect these query patches from the original image.
During the pre-training, we address two critical issues: multi-task learning
and multi-query localization. (1) To trade off classification and localization
preferences in the pretext task, we freeze the CNN backbone and propose a patch
feature reconstruction branch which is jointly optimized with patch detection.
(2) To perform multi-query localization, we introduce UP-DETR from single-query
patch and extend it to multi-query patches with object query shuffle and
attention mask. In our experiments, UP-DETR significantly boosts the
performance of DETR with faster convergence and higher average precision on
object detection, one-shot detection and panoptic segmentation. Code and
pre-training models: this https URL

本研究提出了一种名为随机查询块检测的预训练任务 (UP-DETR)，通过多查询本地化技术和注意力掩码来将其从单一查询块推广到多查询块，提高了检测性能和收敛速度。