Deep Neural Networks (DNN) are nowadays largely adopted in many application
domains thanks to their human-like, or even superhuman, performance in specific
tasks. However, due to unpredictable/unconsidered operating conditions,
unexpected failures show up on field, making the performance of a DNN in
operation very different from the one estimated prior to release. In the life
cycle of DNN systems, the assessment of accuracy is typically addressed in two
ways: offline, via sampling of operational inputs, or online, via
pseudo-oracles. The former is considered more expensive due to the need for
manual labeling of the sampled inputs. The latter is automatic but less
accurate. We believe that emerging iterative industrial-strength life cycle
models for Machine Learning systems, like MLOps, offer the possibility to
leverage inputs observed in operation not only to provide faithful estimates of
a DNN accuracy, but also to improve it through remodeling/retraining actions.
We propose DAIC (DNN Assessment and Improvement Cycle), an approach which
combines ''low-cost'' online pseudo-oracles and ''high-cost'' offline sampling
techniques to estimate and improve the operational accuracy of a DNN in the
iterations of its life cycle. Preliminary results show the benefits of
combining the two approaches and integrating them in the DNN life cycle.

我们提出了 DAIC（DNN 评估和改进周期）方法，结合 “低成本” 在线伪预测和 “高成本” 离线抽样技术来估算和提高 DNN 系统的操作准确性，并将它们整合到 DNN 系统的迭代生命周期中。