The training paradigm of DETRs is heavily contingent upon pre-training their
backbone on the ImageNet dataset. However, the limited supervisory signals
provided by the image classification task and one-to-one matching strategy
result in an inadequately pre-trained neck for DETRs. Additionally, the
instability of matching in the early stages of training engenders
inconsistencies in the optimization objectives of DETRs. To address these
issues, we have devised an innovative training methodology termed step-by-step
training. Specifically, in the first stage of training, we employ a classic
detector, pre-trained with a one-to-many matching strategy, to initialize the
backbone and neck of the end-to-end detector. In the second stage of training,
we froze the backbone and neck of the end-to-end detector, necessitating the
training of the decoder from scratch. Through the application of step-by-step
training, we have introduced the first real-time end-to-end object detection
model that utilizes a purely convolutional structure encoder, DETR with YOLO
(DEYO). Without reliance on any supplementary training data, DEYO surpasses all
existing real-time object detectors in both speed and accuracy. Moreover, the
comprehensive DEYO series can complete its second-phase training on the COCO
dataset using a single 8GB RTX 4060 GPU, significantly reducing the training
expenditure. Source code and pre-trained models are available at
this https URL

通过逐步训练方法，我们介绍了第一个实时端到端的目标检测模型 DEYO，它利用了纯卷积结构编码器，并且在速度和准确性上超过了现有的实时目标检测器。此外，通过仅使用单个 8GB RTX 4060 GPU，DEYO 系列可以在 COCO 数据集上完成第二阶段的训练，从而显著降低了训练成本。

DEYO：用 YOLO 实现的端到端物体检测的 DETR

DEYO: DETR with YOLO for End-to-End Object Detection

Recently, end-to-end object detectors have gained significant attention from
the research community due to their outstanding performance. However, DETR
typically relies on supervised pretraining of the backbone on ImageNet, which
limits the practical application of DETR and the design of the backbone,
affecting the model's potential generalization ability. In this paper, we
propose a new training method called step-by-step training. Specifically, in
the first stage, the one-to-many pre-trained YOLO detector is used to
initialize the end-to-end detector. In the second stage, the backbone and
encoder are consistent with the DETR-like model, but only the detector needs to
be trained from scratch. Due to this training method, the object detector does
not need the additional dataset (ImageNet) to train the backbone, which makes
the design of the backbone more flexible and dramatically reduces the training
cost of the detector, which is helpful for the practical application of the
object detector. At the same time, compared with the DETR-like model, the
step-by-step training method can achieve higher accuracy than the traditional
training method of the DETR-like model. With the aid of this novel training
method, we propose a brand-new end-to-end real-time object detection model
called DEYOv3. DEYOv3-N achieves 41.1% on COCO val2017 and 270 FPS on T4 GPU,
while DEYOv3-L achieves 51.3% AP and 102 FPS. Without the use of additional
training data, DEYOv3 surpasses all existing real-time object detectors in
terms of both speed and accuracy. It is worth noting that for models of N, S,
and M scales, the training on the COCO dataset can be completed using a single
24GB RTX3090 GPU.

提出了一种名为 DEYOv3 的全新端到端实时目标检测模型，采用逐步训练方法，设计灵活、训练成本低，速度和准确性均优于现有的实时目标检测器。