Albeit revealing impressive predictive performance for several computer
vision tasks, deep neural networks (DNNs) are prone to making overconfident
predictions. This limits the adoption and wider utilization of DNNs in many
safety-critical applications. There have been recent efforts toward calibrating
DNNs, however, almost all of them focus on the classification task.
Surprisingly, very little attention has been devoted to calibrating modern
DNN-based object detectors, especially detection transformers, which have
recently demonstrated promising detection performance and are influential in
many decision-making systems. In this work, we address the problem by proposing
a mechanism for calibrated detection transformers (Cal-DETR), particularly for
Deformable-DETR, UP-DETR and DINO. We pursue the train-time calibration route
and make the following contributions. First, we propose a simple yet effective
approach for quantifying uncertainty in transformer-based object detectors.
Second, we develop an uncertainty-guided logit modulation mechanism that
leverages the uncertainty to modulate the class logits. Third, we develop a
logit mixing approach that acts as a regularizer with detection-specific losses
and is also complementary to the uncertainty-guided logit modulation technique
to further improve the calibration performance. Lastly, we conduct extensive
experiments across three in-domain and four out-domain scenarios. Results
corroborate the effectiveness of Cal-DETR against the competing train-time
methods in calibrating both in-domain and out-domain detections while
maintaining or even improving the detection performance. Our codebase and
pre-trained models can be accessed at
https://github.com/akhtarvision/cal-detr.

提出一种用于校准检测变压器的机制，以减少深度神经网络在目标检测中的错误率并提高其校准性能。

Cal-DETR: 分类校准检测器

Cal-DETR: Calibrated Detection Transformer

Physical adversarial attacks have put a severe threat to DNN-based object
detectors. To enhance security, a combination of visible and infrared sensors
is deployed in various scenarios, which has proven effective in disabling
existing single-modal physical attacks. To further demonstrate the potential
risks in such cases, we design a unified adversarial patch that can perform
cross-modal physical attacks, achieving evasion in both modalities
simultaneously with a single patch. Given the different imaging mechanisms of
visible and infrared sensors, our work manipulates patches' shape features,
which can be captured in different modalities when they undergo changes. To
deal with challenges, we propose a novel boundary-limited shape optimization
approach that aims to achieve compact and smooth shapes for the adversarial
patch, making it easy to implement in the physical world. And a score-aware
iterative evaluation method is also introduced to balance the fooling degree
between visible and infrared detectors during optimization, which guides the
adversarial patch to iteratively reduce the predicted scores of the multi-modal
sensors. Furthermore, we propose an Affine-Transformation-based enhancement
strategy that makes the learnable shape robust to various angles, thus
mitigating the issue of shape deformation caused by different shooting angles
in the real world. Our method is evaluated against several state-of-the-art
object detectors, achieving an Attack Success Rate (ASR) of over 80%. We also
demonstrate the effectiveness of our approach in physical-world scenarios under
various settings, including different angles, distances, postures, and scenes
for both visible and infrared sensors.

通过可见光和红外线传感器的组合，我们设计了一种统一的对抗补丁，能够在不同模态下实施物理攻击，通过改变补丁的形状特征来规避可见光和红外线传感器的检测，实现了同时攻击两种模态的目标检测系统。