Ensemble learning combines several individual models to obtain better generalization performance. In this work we present a practical method for estimating the joint power of several classifiers which differs from existing approaches by {\em not relying on labels}, hence enabling the work in unsupervised setting of huge datasets. It differs from existing methods which define a "diversity measure". The heart of the method is a combinatorial bound on the number of mistakes the ensemble is likely to make. The bound can be efficiently approximated in time linear in the number of samples. Thus allowing an efficient search for a combination of classifiers that are likely to produce higher joint accuracy. Moreover, having the bound applicable to unlabeled data makes it both accurate and practical in modern setting of unsupervised learning. We demonstrate the method on popular large-scale face recognition datasets which provide a useful playground for fine-grain classification tasks using noisy data over many classes. The proposed framework fits neatly in trending practices of unsupervised learning. It is a measure of the inherent independence of a set of classifiers not relying on extra information such as another classifier or labeled data.

本文提出一种实用的方法来估计多个分类器的联合能力，与现有方法不同的是它不依赖于标签，从而使得在大型无监督数据集中进行工作成为可能。该方法通过组合误差的组合边界实现，能够高效地搜索可能产生更高联合准确性的分类器组合。此外，该方法适用于无标签数据，使其在现代的无监督学习场景中既准确又实用。通过在流行的大规模人脸识别数据集上进行实验证明了该方法的有效性，这些数据集为使用嘈杂数据进行细粒度分类任务提供了一个有用的测试平台。该方法与无监督学习的趋势性实践相吻合，通过衡量一组分类器的内在独立性，而不依赖于其他分类器或标记数据。

集合准确性的实用估计