The application of Deep Neural Networks (DNNs) to a broad variety of tasks
demands methods for coping with the complex and opaque nature of these
architectures. When a gold standard is available, performance assessment treats
the DNN as a black box and computes standard metrics based on the comparison of
the predictions with the ground truth. A deeper understanding of performances
requires going beyond such evaluation metrics to diagnose the model behavior
and the prediction errors. This goal can be pursued in two complementary ways.
On one side, model interpretation techniques "open the box" and assess the
relationship between the input, the inner layers and the output, so as to
identify the architecture modules most likely to cause the performance loss. On
the other hand, black-box error diagnosis techniques study the correlation
between the model response and some properties of the input not used for
training, so as to identify the features of the inputs that make the model
fail. Both approaches give hints on how to improve the architecture and/or the
training process. This paper focuses on the application of DNNs to Computer
Vision (CV) tasks and presents a survey of the tools that support the black-box
performance diagnosis paradigm. It illustrates the features and gaps of the
current proposals, discusses the relevant research directions and provides a
brief overview of the diagnosis tools in sectors other than CV.

本论文调查了支持黑匣子性能诊断范式的工具，以深度神经网络为基础，探讨了模型行为和预测误差的诊断过程，并讨论了相关研究方向和除 CV 外的诊断工具的简要概述。