Zero-Shot Learning (ZSL), which aims at automatically recognizing unseen
objects, is a promising learning paradigm to understand new real-world
knowledge for machines continuously. Recently, the Knowledge Graph (KG) has
been proven as an effective scheme for handling the zero-shot task with
large-scale and non-attribute data. Prior studies always embed relationships of
seen and unseen objects into visual information from existing knowledge graphs
to promote the cognitive ability of the unseen data. Actually, real-world
knowledge is naturally formed by multimodal facts. Compared with ordinary
structural knowledge from a graph perspective, multimodal KG can provide
cognitive systems with fine-grained knowledge. For example, the text
description and visual content can depict more critical details of a fact than
only depending on knowledge triplets. Unfortunately, this multimodal
fine-grained knowledge is largely unexploited due to the bottleneck of feature
alignment between different modalities. To that end, we propose a multimodal
intensive ZSL framework that matches regions of images with corresponding
semantic embeddings via a designed dense attention module and self-calibration
loss. It makes the semantic transfer process of our ZSL framework learns more
differentiated knowledge between entities. Our model also gets rid of the
performance limitation of only using rough global features. We conduct
extensive experiments and evaluate our model on large-scale real-world data.
The experimental results clearly demonstrate the effectiveness of the proposed
model in standard zero-shot classification tasks.

本论文针对零样本学习任务中多模态的问题，提出了一种基于知识图谱和密集注意力模块的多模态零样本学习框架，实现了对实体更具差异化的语义迁移过程和精细化的知识获取。