Generalized Zero-Shot Learning (GZSL) identifies unseen categories by
knowledge transferred from the seen domain, relying on the intrinsic
interactions between visual and semantic information. Prior works mainly
localize regions corresponding to the sharing attributes. When various visual
appearances correspond to the same attribute, the sharing attributes inevitably
introduce semantic ambiguity, hampering the exploration of accurate
semantic-visual interactions. In this paper, we deploy the dual semantic-visual
transformer module (DSVTM) to progressively model the correspondences between
attribute prototypes and visual features, constituting a progressive
semantic-visual mutual adaption (PSVMA) network for semantic disambiguation and
knowledge transferability improvement. Specifically, DSVTM devises an
instance-motivated semantic encoder that learns instance-centric prototypes to
adapt to different images, enabling the recast of the unmatched semantic-visual
pair into the matched one. Then, a semantic-motivated instance decoder
strengthens accurate cross-domain interactions between the matched pair for
semantic-related instance adaption, encouraging the generation of unambiguous
visual representations. Moreover, to mitigate the bias towards seen classes in
GZSL, a debiasing loss is proposed to pursue response consistency between seen
and unseen predictions. The PSVMA consistently yields superior performances
against other state-of-the-art methods. Code will be available at:
this https URL

本文提出了双重语义视觉变换器模块 (DSVTM) 来逐步建立属性原型和视觉特征之间的对应关系，构成了一种逐步的语义 - 视觉相互适应 (PSVMA) 网络，以实现语义消歧和知识可转移性的提高。DSVTM 设计了一种实例驱动的语义编码器，可以学习到面向不同图像的实例中心原型，在匹配语义 - 视觉对时可以重新配对为匹配的对。随后，语义驱动的实例解码器加强了匹配对之间精确的跨域交互，以推动无歧义视觉表示的生成。此外，为了缓解 GZSL 中对已知类的偏见，提出了减轻偏见的损失以追求已知和未知的预测一致性。PSVMA 表现出对其他最先进的方法具有卓越的性能。