Creating resilient machine learning (ML) systems has become necessary to
ensure production-ready ML systems that acquire user confidence seamlessly. The
quality of the input data and the model highly influence the successful
end-to-end testing in data-sensitive systems. However, the testing approaches
of input data are not as systematic and are few compared to model testing. To
address this gap, this paper presents the Fault Injection for Undesirable
Learning in input Data (FIUL-Data) testing framework that tests the resilience
of ML models to multiple intentionally-triggered data faults. Data mutators
explore vulnerabilities of ML systems against the effects of different fault
injections. The proposed framework is designed based on three main ideas: The
mutators are not random; one data mutator is applied at an instance of time,
and the selected ML models are optimized beforehand. This paper evaluates the
FIUL-Data framework using data from analytical chemistry, comprising retention
time measurements of anti-sense oligonucleotide. Empirical evaluation is
carried out in a two-step process in which the responses of selected ML models
to data mutation are analyzed individually and then compared with each other.
The results show that the FIUL-Data framework allows the evaluation of the
resilience of ML models. In most experiments cases, ML models show higher
resilience at larger training datasets, where gradient boost performed better
than support vector regression in smaller training sets. Overall, the mean
squared error metric is useful in evaluating the resilience of models due to
its higher sensitivity to data mutation.

创建弹性机器学习系统对于确保生产就绪的机器学习系统并顺利获取用户信任已成为必要。输入数据和模型的质量对数据敏感系统的端到端测试成功具有高度影响。然而，与模型测试相比，输入数据的测试方法缺乏系统性且较少。为了填补这一空白，本文提出了一种基于故障注入技术的输入数据故障注入测试框架（FIUL-Data），该框架测试机器学习模型对多种故意引发的数据故障的弹性。数据变异器探索机器学习系统的脆弱性，针对不同故障注入效果。提出的框架基于三个主要想法设计：变异器不是随机的；在某个时刻应用一个数据变异器；事先优化选定的机器学习模型。本文使用来自分析化学的数据进行了 FIUL-Data 框架的实证评估，包括反义寡核苷酸保留时间测量。经验评估分为两个步骤进行，首先分析选定机器学习模型对数据变异的响应，然后对比彼此。结果显示 FIUL-Data 框架允许评估机器学习模型的弹性。在大多数实验情况下，较大的训练数据集上，机器学习模型表现出更高的弹性，在较小的训练集中梯度提升方法优于支持向量回归。总体而言，均方误差指标在评估模型弹性方面具有较高的敏感性。