Many approaches in generalized zero-shot learning rely on cross-modal mapping between the image feature space and the class embedding space. As labeled images are rare, one direction is to augment the dataset by generating either images or image features. However, the former misses fine-grained details and the latter requires learning a mapping associated with class embeddings. In this work, we take feature generation one step further and propose a model where a shared latent space of image features and class embeddings is learned by modality-specific aligned variational autoencoders. This leaves us with the required discriminative information about the image and classes in the latent features, on which we train a softmax classifier. The key to our approach is that we align the distributions learned from images and from side-information to construct latent features that contain the essential multi-modal information associated with unseen classes. We evaluate our learned latent features on several benchmark datasets, i.e. CUB, SUN, AWA1 and AWA2, and establish a new state-of-the-art on generalized zero-shot as well as on few-shot learning. Moreover, our results on ImageNet with various zero-shot splits show that our latent features generalize well in large-scale settings.

本研究提出了一种基于变分自编码器的模型，通过学习图像特征和类别嵌入的共享潜在空间生成潜在特征，并在其中训练softmax分类器，以构建包含未知类别的本质多模态信息的潜在特征。在多个基准数据集上的实验表明，该方法在零样本学习和少样本学习方面取得了最新的最佳表现，并且在各种零样本分割的ImageNet上，该方法的潜在特征可以在大规模的情况下进行很好的泛化。

通过对齐变分自编码器实现广义零样本和少样本学习