Quantification is the supervised learning task that consists of training
predictors of the class prevalence values of sets of unlabelled data, and is of
special interest when the labelled data on which the predictor has been trained
and the unlabelled data are not IID, i.e., suffer from dataset shift. To date,
quantification methods have mostly been tested only on a special case of
dataset shift, i.e., prior probability shift; the relationship between
quantification and other types of dataset shift remains, by and large,
unexplored. In this work we carry out an experimental analysis of how current
quantification algorithms behave under different types of dataset shift, in
order to identify limitations of current approaches and hopefully pave the way
for the development of more broadly applicable methods. We do this by proposing
a fine-grained taxonomy of types of dataset shift, by establishing protocols
for the generation of datasets affected by these types of shift, and by testing
existing quantification methods on the datasets thus generated. One finding
that results from this investigation is that many existing quantification
methods that had been found robust to prior probability shift are not
necessarily robust to other types of dataset shift. A second finding is that no
existing quantification method seems to be robust enough to dealing with all
the types of dataset shift we simulate in our experiments. The code needed to
reproduce all our experiments is publicly available at
this https URL

在这项研究中，我们通过实验分析了当前的定量化算法在不同类型的数据集偏移下的行为，以识别现有方法的局限性，并为开发更广泛适用的方法铺平道路。我们通过提出细粒度的数据集偏移类型分类，建立受这些偏移类型影响的数据集生成协议，并在生成的数据集上测试现有的定量化方法。研究结果表明，许多已被发现对于先验概率偏移具有鲁棒性的定量化方法对于其他类型的数据集偏移可能不具备鲁棒性。此外，没有发现任何现有的定量化方法能够对我们在实验中模拟的所有数据集偏移类型具备足够的鲁棒性。