Reliability estimation of Machine Learning (ML) models is becoming a crucial
subject. This is particularly the case when such \mbox{models} are deployed in
safety-critical applications, as the decisions based on model predictions can
result in hazardous situations. In this regard, recent research has proposed
methods to achieve safe, \mbox{dependable}, and reliable ML systems. One such
method consists of detecting and analyzing distributional shift, and then
measuring how such systems respond to these shifts. This was proposed in
earlier work in SafeML. This work focuses on the use of SafeML for time series
data, and on reliability and robustness estimation of ML-forecasting methods
using statistical distance measures. To this end, distance measures based on
the Empirical Cumulative Distribution Function (ECDF) proposed in SafeML are
explored to measure Statistical-Distance Dissimilarity (SDD) across time
series. We then propose SDD-based Reliability Estimate (StaDRe) and SDD-based
Robustness (StaDRo) measures. With the help of a clustering technique, the
similarity between the statistical properties of data seen during training and
the forecasts is identified. The proposed method is capable of providing a link
between dataset SDD and Key Performance Indicators (KPIs) of the ML models.

本研究探讨了基于 Empirical Cumulative Distribution Function（ECDF）的距离度量方法，以测量时间序列数据中的 ML - 预测方法的可靠性和鲁棒性估计，提出了 SDD-based 可靠性估计（StaDRe）和 SDD-based 鲁棒性估计（StaDRo）措施，并可提供数据集 SDD 与 ML 模型的关键绩效指标之间的联系。