We present a theory of ensemble diversity, explaining the nature and effect
of diversity for a wide range of supervised learning scenarios. This challenge,
of understanding ensemble diversity, has been referred to as the holy grail of
ensemble learning, an open question for over 30 years. Our framework reveals
that diversity is in fact a hidden dimension in the bias-variance decomposition
of an ensemble. In particular, we prove a family of exact
bias-variance-diversity decompositions, for both classification and regression
losses, e.g., squared, and cross-entropy. The framework provides a methodology
to automatically identify the combiner rule enabling such a decomposition,
specific to the loss. The formulation of diversity is therefore dependent on
just two design choices: the loss, and the combiner. For certain choices (e.g.,
0-1 loss with majority voting) the effect of diversity is necessarily dependent
on the target label. Experiments illustrate how we can use our framework to
understand the diversity-encouraging mechanisms of popular ensemble methods:
Bagging, Boosting, and Random Forests.

本论文提出了一种集成多样性理论，阐明了多样性的本质及对各种监督学习场景的影响，揭示了多样性是集合偏差方差分解中的一个隐含维度，同时提供了一种自动识别可使偏差方差多样性分解成立的组合规则的方法。在此基础上，实验进一步说明了我们可以用该框架来理解当前流行的集成方法。