The current approach for testing the robustness of object detectors suffers from serious deficiencies such as improper methods of performing out-of-distribution detection and using calibration metrics which do not consider both localisation and classification quality. In this work, we address these issues, and introduce the Self-Aware Object Detection (SAOD) task, a unified testing framework which respects and adheres to the challenges that object detectors face in safety-critical environments such as autonomous driving. Specifically, the SAOD task requires an object detector to be: robust to domain shift; obtain reliable uncertainty estimates for the entire scene; and provide calibrated confidence scores for the detections. We extensively use our framework, which introduces novel metrics and large scale test datasets, to test numerous object detectors in two different use-cases, allowing us to highlight critical insights into their robustness performance. Finally, we introduce a simple baseline for the SAOD task, enabling researchers to benchmark future proposed methods and move towards robust object detectors which are fit for purpose. Code is available at https://github.com/fiveai/saod

本文介绍了一个名为SAOD的任务，它是一种统一的测试框架，旨在针对自主驾驶等安全关键环境中物体检测器所面临的挑战，并解决了当前测试物体检测器鲁棒性的方法所存在的问题，并引入了新颖的度量和大规模测试数据集来测试多个物体检测器，最终引入一个简单的基线，以便为研究人员提供基准测试方法。

通过可靠的不确定性量化与校准，构建自我感知的物体检测器