Transformer-based object detectors (DETR) have shown significant performance across machine vision tasks, ultimately in object detection. This detector is based on a self-attention mechanism along with the transformer encoder-decoder architecture to capture the global context in the image. The critical issue to be addressed is how this model architecture can handle different image nuisances, such as occlusion and adversarial perturbations. We studied this issue by measuring the performance of DETR with different experiments and benchmarking the network with convolutional neural network (CNN) based detectors like YOLO and Faster-RCNN. We found that DETR performs well when it comes to resistance to interference from information loss in occlusion images. Despite that, we found that the adversarial stickers put on the image require the network to produce a new unnecessary set of keys, queries, and values, which in most cases, results in a misdirection of the network. DETR also performed poorer than YOLOv5 in the image corruption benchmark. Furthermore, we found that DETR depends heavily on the main query when making a prediction, which leads to imbalanced contributions between queries since the main query receives most of the gradient flow.

基于Transformer的目标检测器(DETR)在机器视觉任务中表现出显著性能，但其在处理遮挡和对抗扰动等不同图像干扰方面存在问题。我们通过多种实验和将DETR与基于卷积神经网络(CNN)的检测器（如YOLO和Faster-RCNN）进行基准测试来研究这个问题。我们发现DETR在处理遮挡图像的信息丢失干扰方面表现良好。然而，在涂有对抗标记的图像上，网络需要产生一组新的不必要的键、查询和值，导致网络方向错误。与图像损坏基准测试中YOLOv5相比，DETR的性能也较差。此外，我们发现DETR在进行预测时严重依赖于主要查询，导致查询之间的贡献不平衡，因为主要查询接收大部分梯度流。

探索Difficult Images对检测变换器（DETR）的鲁棒性和特性