We study boosting algorithms from a new perspective. We show that the Lagrange dual problems of AdaBoost, LogitBoost and soft-margin LPBoost with generalized hinge loss are all entropy maximization problems. By looking at the dual problems of these boosting algorithms, we show that the success of boosting algorithms can be understood in terms of maintaining a better margin distribution by maximizing margins and at the same time controlling the margin variance.We also theoretically prove that, approximately, AdaBoost maximizes the average margin, instead of the minimum margin. The duality formulation also enables us to develop column generation based optimization algorithms, which are totally corrective. We show that they exhibit almost identical classification results to that of standard stage-wise additive boosting algorithms but with much faster convergence rates. Therefore fewer weak classifiers are needed to build the ensemble using our proposed optimization technique.

研究使用新视角的提升算法，证明 AdaBoost、LogitBoost 和软边界 LPBoost 的拉格朗日对偶问题都是熵最大化问题，并通过研究这些算法的对偶问题，表明了提升算法的成功可以从最大化边缘并同时控制边缘方差的角度来理解。通过列生成优化算法，实现了更快的收敛率，并使得使用提议的优化技术建立集成所需的弱分类器更少。

提升算法的双重表述