Compositional zero-shot learning (CZSL) aims to recognize unseen compositions
with prior knowledge of known primitives (attribute and object). Previous works
for CZSL often suffer from grasping the contextuality between attribute and
object, as well as the discriminability of visual features, and the long-tailed
distribution of real-world compositional data. We propose a simple and scalable
framework called Composition Transformer (CoT) to address these issues. CoT
employs object and attribute experts in distinctive manners to generate
representative embeddings, using the visual network hierarchically. The object
expert extracts representative object embeddings from the final layer in a
bottom-up manner, while the attribute expert makes attribute embeddings in a
top-down manner with a proposed object-guided attention module that models
contextuality explicitly. To remedy biased prediction caused by imbalanced data
distribution, we develop a simple minority attribute augmentation (MAA) that
synthesizes virtual samples by mixing two images and oversampling minority
attribute classes. Our method achieves SoTA performance on several benchmarks,
including MIT-States, C-GQA, and VAW-CZSL. We also demonstrate the
effectiveness of CoT in improving visual discrimination and addressing the
model bias from the imbalanced data distribution. The code is available at
this https URL

通过使用 Composition Transformer（CoT）框架，我们提出了一种简单可扩展的方法来解决组合式零样本学习（CZSL）中的上下文问题、视觉特征可辨识性问题和真实世界组合数据的长尾分布问题；CoT 框架通过底层的自下而上方式提取代表性的物体嵌入，并通过顶层的自上而下方式以一种显式建模上下文性的对象引导注意力模块生成代表性的属性嵌入；我们还开发了一种简单的少数类属性增广（MAA）方法，通过混合两张图像和过采样少数类属性来合成虚拟样本，以解决不平衡的数据分布所引起的预测偏差；我们的方法在多个基准测试集上达到了最先进的性能，包括 MIT-States，C-GQA 和 VAW-CZSL；此外，我们还展示了 CoT 在提高视觉辨识能力和解决因不平衡数据分布而引起的模型偏差方面的有效性。