The effectiveness of active learning largely depends on the sampling
efficiency of the acquisition function. Expected Loss Reduction (ELR) focuses
on a Bayesian estimate of the reduction in classification error, and more
general costs fit in the same framework. We propose Bayesian Estimate of Mean
Proper Scores (BEMPS) to estimate the increase in strictly proper scores such
as log probability or negative mean square error within this framework. We also
prove convergence results for this general class of costs. To facilitate better
experimentation with the new acquisition functions, we develop a complementary
batch AL algorithm that encourages diversity in the vector of expected changes
in scores for unlabeled data. To allow high-performance classifiers, we combine
deep ensembles, and dynamic validation set construction on pretrained models,
and further speed up the ensemble process with the idea of Monte Carlo Dropout.
Extensive experiments on both texts and images show that the use of mean square
error and log probability with BEMPS yields robust acquisition functions and
well-calibrated classifiers, and consistently outperforms the others tested.
The advantages of BEMPS over the others are further supported by a set of
qualitative analyses, where we visualise their sampling behaviour using data
maps and t-SNE plots.

提出了一种基于贝叶斯估计的增加严格适当得分的方法 BEMPS，用于在主动学习中估计严格适当得分的增加，该方法在实验证明了其较其他方法更强大和更具鲁棒性，并通过定性分析得到了支持。