Flaky tests can pass or fail non-deterministically, without alterations to a software system. Such tests are frequently encountered by developers and hinder the credibility of test suites. State-of-the-art research incorporates machine learning solutions into flaky test detection and achieves reasonably good accuracy. Moreover, the majority of automated flaky test repair solutions are designed for specific types of flaky tests. This research work proposes a novel categorization framework, called FlaKat, which uses machine-learning classifiers for fast and accurate prediction of the category of a given flaky test that reflects its root cause. Sampling techniques are applied to address the imbalance between flaky test categories in the International Dataset of Flaky Test (IDoFT). A new evaluation metric, called Flakiness Detection Capacity (FDC), is proposed for measuring the accuracy of classifiers from the perspective of information theory and provides proof for its effectiveness. The final FDC results are also in agreement with F1 score regarding which classifier yields the best flakiness classification.

该研究提出了一个名为FlaKat的新分类框架，利用机器学习分类器快速准确地预测给定的易崩溃测试的类别，反映其根本原因，并提出了一种新的评估指标FDC，用于从信息论的角度衡量分类器的准确性。最终的FDC结果与F1得分一致，确定了能够产生最佳易崩溃分类的分类器。

FlaKat: 一个基于机器学习的一致性测试分类框架