We propose a method, called Label Embedding Network, which can learn label
representation (label embedding) during the training process of deep networks.
With the proposed method, the label embedding is adaptively and automatically
learned through back propagation. The original one-hot represented loss
function is converted into a new loss function with soft distributions, such
that the originally unrelated labels have continuous interactions with each
other during the training process. As a result, the trained model can achieve
substantially higher accuracy and with faster convergence speed. Experimental
results based on competitive tasks demonstrate the effectiveness of the
proposed method, and the learned label embedding is reasonable and
interpretable. The proposed method achieves comparable or even better results
than the state-of-the-art systems. The source code is available at
https://github.com/lancopku/LabelEmb.

本研究提出了一种标签嵌入网络的方法，在深度网络的训练过程中学习标签表示（标签嵌入）。通过反向传播，该方法可以自适应地自动地学习标签嵌入，将原来的单热表示丢失函数转换成带有软分布的新的丢失函数，使得原来不相关的标签在训练过程中具有连续的相互作用。实验结果表明，所提出的方法可以在竞争任务中获得更高的准确性和更快的收敛速度，并且所学习的标签嵌入是合理且可解释的。