Humans are able to identify and categorize novel compositions of known
concepts. The task in Compositional Zero-Shot learning (CZSL) is to learn
composition of primitive concepts, i.e. objects and states, in such a way that
even their novel compositions can be zero-shot classified. In this work, we do
not assume any prior knowledge on the feasibility of novel compositions
i.e.open-world setting, where infeasible compositions dominate the search
space. We propose a Compositional Variational Graph Autoencoder (CVGAE)
approach for learning the variational embeddings of the primitive concepts
(nodes) as well as feasibility of their compositions (via edges). Such
modelling makes CVGAE scalable to real-world application scenarios. This is in
contrast to SOTA method, CGE, which is computationally very expensive. e.g.for
benchmark C-GQA dataset, CGE requires 3.94 x 10^5 nodes, whereas CVGAE requires
only 1323 nodes. We learn a mapping of the graph and image embeddings onto a
common embedding space. CVGAE adopts a deep metric learning approach and learns
a similarity metric in this space via bi-directional contrastive loss between
projected graph and image embeddings. We validate the effectiveness of our
approach on three benchmark datasets.We also demonstrate via an image retrieval
task that the representations learnt by CVGAE are better suited for
compositional generalization.

本文提出了一种基于组合变分图自编码器（CVGAE）的方法来学习基本概念及其组成的可行性，以达到零样本分类的目的，并通过图像检索验证了该方法的有效性。

基于变分图嵌入的开放式世界组合零样本学习

On Leveraging Variational Graph Embeddings for Open World Compositional Zero-Shot Learning

In this paper, we study whether representations of primitive concepts--such
as colors and shapes of object parts--emerge automatically within these
pretrained VL models. We propose a two-step framework, Compositional Concept
Mapping (CompMap), to investigate this. CompMap asks a VL model to generate
concept activations with text prompts from a predefined list of primitive
concepts, and then learns to construct an explicit composition model that maps
the primitive concept activations (e.g. the likelihood of black tail or red
wing) to composite concepts (e.g. a red-winged blackbird). We demonstrate that
a composition model can be designed as a set operation, and show that a
composition model is straightforward for machines to learn from ground truth
primitive concepts (as a linear classifier). We thus hypothesize that if
primitive concepts indeed emerge in a VL pretrained model, its primitive
concept activations can be used to learn a composition model similar to the one
designed by experts. We propose a quantitative metric to measure the degree of
similarity, and refer to the metric as the interpretability of the learned
primitive concept representations of VL models. We also measure the
classification accuracy when using the primitive concept activations and the
learned composition model to predict the composite concepts, and refer to it as
the usefulness metric. Our study reveals that state-of-the-art VL pretrained
models learn primitive concepts that are highly useful for fine-grained visual
recognition on the CUB dataset, and compositional generalization tasks on the
MIT-States dataset. However, we observe that the learned composition models
have low interpretability in our qualitative analyses. Our results reveal the
limitations of existing VL models, and the necessity of pretraining objectives
that encourage the acquisition of primitive concepts.

本文研究预训练可视化学习模型是否会自动产生原始概念的表示，例如物体部分的颜色和形状，提出了一个用于探究这个问题的两步框架：组合概念映射 (CompMap)，证明组合模型可以被设计为一组操作，并发现用基本概念激活的模型可以用于细粒度视觉识别和组合泛化任务。