Dense retrieval is widely used for entity linking to retrieve entities from
large-scale knowledge bases. Mainstream techniques are based on a dual-encoder
framework, which encodes mentions and entities independently and calculates
their relevances via rough interaction metrics, resulting in difficulty in
explicitly modeling multiple mention-relevant parts within entities to match
divergent mentions. Aiming at learning entity representations that can match
divergent mentions, this paper proposes a Multi-View Enhanced Distillation
(MVD) framework, which can effectively transfer knowledge of multiple
fine-grained and mention-relevant parts within entities from cross-encoders to
dual-encoders. Each entity is split into multiple views to avoid irrelevant
information being over-squashed into the mention-relevant view. We further
design cross-alignment and self-alignment mechanisms for this framework to
facilitate fine-grained knowledge distillation from the teacher model to the
student model. Meanwhile, we reserve a global-view that embeds the entity as a
whole to prevent dispersal of uniform information. Experiments show our method
achieves state-of-the-art performance on several entity linking benchmarks.

本篇论文提出了一种多视图增强蒸馏框架，通过跨编解码器到对编解码器的多精细和提及相关部分的知识传输，从而有效地学习与不同提及相匹配的实体表示，实验表明该方法在几个实体链接基准上实现了最优性能。