The standard loss function used to train neural network classifiers, categorical cross-entropy (CCE), seeks to maximize accuracy on the training data; building useful representations is not a necessary byproduct of this objective. In this work, we propose clustering-oriented representation learning (COREL) as an alternative to CCE in the context of a generalized attractive-repulsive loss framework. COREL has the consequence of building latent representations that collectively exhibit the quality of natural clustering within the latent space of the final hidden layer, according to a predefined similarity function. Despite being simple to implement, COREL variants outperform or perform equivalently to CCE in a variety of scenarios, including image and news article classification using both feed-forward and convolutional neural networks. Analysis of the latent spaces created with different similarity functions facilitates insights on the different use cases COREL variants can satisfy, where the Cosine-COREL variant makes a consistently clusterable latent space, while Gaussian-COREL consistently obtains better classification accuracy than CCE.

本文提出了聚类导向的表示学习(COREL)作为分类器的一个替代方法，可以建立起了一组高质量自然聚类的潜在表示。经过分析，不同的相似性函数可以满足不同的使用场景，其中余弦-COREL变体形成了一个一致的可聚类的潜在空间，而高斯-COREL变体在分类准确度上优于CCE。

基于聚类的具有吸引-排斥损失的表示学习