Motivated by that DETR-based approaches have established new records on COCO
detection and segmentation benchmarks, many recent endeavors show increasing
interest in how to further improve DETR-based approaches by pre-training the
Transformer in a self-supervised manner while keeping the backbone frozen. Some
studies already claimed significant improvements in accuracy. In this paper, we
take a closer look at their experimental methodology and check if their
approaches are still effective on the very recent state-of-the-art such as
$\mathcal{H}$-Deformable-DETR. We conduct thorough experiments on COCO object
detection tasks to study the influence of the choice of pre-training datasets,
localization, and classification target generation schemes. Unfortunately, we
find the previous representative self-supervised approach such as DETReg, fails
to boost the performance of the strong DETR-based approaches on full data
regimes. We further analyze the reasons and find that simply combining a more
accurate box predictor and Objects$365$ benchmark can significantly improve the
results in follow-up experiments. We demonstrate the effectiveness of our
approach by achieving strong object detection results of AP=$59.3\%$ on COCO
val set, which surpasses $\mathcal{H}$-Deformable-DETR + Swin-L by +$1.4\%$.
Last, we generate a series of synthetic pre-training datasets by combining the
very recent image-to-text captioning models (LLaVA) and text-to-image
generative models (SDXL). Notably, pre-training on these synthetic datasets
leads to notable improvements in object detection performance. Looking ahead,
we anticipate substantial advantages through the future expansion of the
synthetic pre-training dataset.

通过对最新的自我监督训练方法进行实验，发现之前的代表性自我监督方法无法提升强 DET-based 方法在完整数据范围上的性能，但通过结合更准确的盒子预测器和 Objects365 基准可以显著提高后续实验的结果，在 COCO 验证集上实现了 AP=59.3% 的强大目标检测结果，超过了 H-Deformable-DETR + Swin-L 的 1.4%；此外，通过合成的预训练数据集（LLaVA 和 SDXL 的组合）进行预训练，可以显著提高目标检测性能，并且在未来扩展合成预训练数据集方面有巨大优势。

重新审视 DETR 目标检测的预训练方法

Revisiting DETR Pre-training for Object Detection

Recently, the dominant DETR-based approaches apply central-concept spatial
prior to accelerate Transformer detector convergency. These methods gradually
refine the reference points to the center of target objects and imbue object
queries with the updated central reference information for spatially
conditional attention. However, centralizing reference points may severely
deteriorate queries' saliency and confuse detectors due to the indiscriminative
spatial prior. To bridge the gap between the reference points of salient
queries and Transformer detectors, we propose SAlient Point-based DETR
(SAP-DETR) by treating object detection as a transformation from salient points
to instance objects. In SAP-DETR, we explicitly initialize a query-specific
reference point for each object query, gradually aggregate them into an
instance object, and then predict the distance from each side of the bounding
box to these points. By rapidly attending to query-specific reference region
and other conditional extreme regions from the image features, SAP-DETR can
effectively bridge the gap between the salient point and the query-based
Transformer detector with a significant convergency speed. Our extensive
experiments have demonstrated that SAP-DETR achieves 1.4 times convergency
speed with competitive performance. Under the standard training scheme,
SAP-DETR stably promotes the SOTA approaches by 1.0 AP. Based on ResNet-DC-101,
SAP-DETR achieves 46.9 AP.

提出基于显著点的 DETR 模型以解决传统模型中参考点集中问题，显著点作为检测器，具有更强的区分能力和更快的收敛速度，实验证明其性能优于 SOTA 方法。