Ensemble methods combine the predictions of several base models. We study
whether or not including more models in an ensemble always improve its average
performance. Such a question depends on the kind of ensemble considered, as
well as the predictive metric chosen. We focus on situations where all members
of the ensemble are a priori expected to perform as well, which is the case of
several popular methods like random forests or deep ensembles. In this setting,
we essentially show that ensembles are getting better all the time if, and only
if, the considered loss function is convex. More precisely, in that case, the
average loss of the ensemble is a decreasing function of the number of models.
When the loss function is nonconvex, we show a series of results that can be
summarised by the insight that ensembles of good models keep getting better,
and ensembles of bad models keep getting worse. To this end, we prove a new
result on the monotonicity of tail probabilities that may be of independent
interest. We illustrate our results on a simple machine learning problem
(diagnosing melanomas using neural nets).

集成方法的性能在预测度量与损失函数的选择下是否随着包含更多模型而提升得到了研究。我们发现，当考虑到损失函数的凸性时，集成的平均损失随着模型数量的增加而降低，而当损失函数是非凸的时候，优秀模型的集成性能提升，而糟糕模型的集成性能下降。