Dimensionality reduction (DR) is often used as a preprocessing step in
classification, but usually one first fixes the DR mapping, possibly using
label information, and then learns a classifier (a filter approach). Best
performance would be obtained by optimizing the classification error jointly
over DR mapping and classifier (a wrapper approach), but this is a difficult
nonconvex problem, particularly with nonlinear DR. Using the method of
auxiliary coordinates, we give a simple, efficient algorithm to train a
combination of nonlinear DR and a classifier, and apply it to a RBF mapping
with a linear SVM. This alternates steps where we train the RBF mapping and a
linear SVM as usual regression and classification, respectively, with a
closed-form step that coordinates both. The resulting nonlinear low-dimensional
classifier achieves classification errors competitive with the state-of-the-art
but is fast at training and testing, and allows the user to trade off runtime
for classification accuracy easily. We then study the role of nonlinear DR in
linear classification, and the interplay between the DR mapping, the number of
latent dimensions and the number of classes. When trained jointly, the DR
mapping takes an extreme role in eliminating variation: it tends to collapse
classes in latent space, erasing all manifold structure, and lay out class
centroids so they are linearly separable with maximum margin.

使用辅助坐标方法，我们提出了一种简单，高效的算法来训练非线性 DR 和分类器的组合，并将其应用于具有线性 SVM 的 RBF 映射。该非线性低维分类器的分类错误率竞争力与最先进技术相当，但在训练和测试过程中非常快速，并允许用户轻松权衡运行时间和分类精度。同时，我们还研究了非线性 DR 在线性分类中的作用以及 DR 映射，潜在维度数量和类别数量之间的相互作用。