Developing an optimal PAC learning algorithm in the realizable setting, where
empirical risk minimization (ERM) is suboptimal, was a major open problem in
learning theory for decades. The problem was finally resolved by Hanneke a few
years ago. Unfortunately, Hanneke's algorithm is quite complex as it returns
the majority vote of many ERM classifiers that are trained on carefully
selected subsets of the data. It is thus a natural goal to determine the
simplest algorithm that is optimal. In this work we study the arguably simplest
algorithm that could be optimal: returning the majority vote of three ERM
classifiers. We show that this algorithm achieves the optimal in-expectation
bound on its error which is provably unattainable by a single ERM classifier.
Furthermore, we prove a near-optimal high-probability bound on this algorithm's
error. We conjecture that a better analysis will prove that this algorithm is
in fact optimal in the high-probability regime.

研究理论以确定最简单的算法，通过选取数据的子集经验风险最小化（ERM）训练而返回多个 ERM 分类器的多数投票，实现高概率区间的最优算法。