Taxonomy expansion task is essential in organizing the ever-increasing volume
of new concepts into existing taxonomies. Most existing methods focus
exclusively on using textual semantics, leading to an inability to generalize
to unseen terms and the "Prototypical Hypernym Problem." In this paper, we
propose Visual Taxonomy Expansion (VTE), introducing visual features into the
taxonomy expansion task. We propose a textual hypernymy learning task and a
visual prototype learning task to cluster textual and visual semantics. In
addition to the tasks on respective modalities, we introduce a hyper-proto
constraint that integrates textual and visual semantics to produce fine-grained
visual semantics. Our method is evaluated on two datasets, where we obtain
compelling results. Specifically, on the Chinese taxonomy dataset, our method
significantly improves accuracy by 8.75 %. Additionally, our approach performs
better than ChatGPT on the Chinese taxonomy dataset.

本文提出了 “视觉分类法扩展”（VTE），将视觉特征引入分类法扩展任务。我们提出了文本上位词学习任务和视觉原型学习任务，以聚类文本和视觉语义。除了各自的任务外，我们还引入了超原型约束，整合文本和视觉语义以产生细粒度的视觉语义。在两个数据集上对我们的方法进行评估，我们获得了令人信服的结果。特别在中文分类法数据集上，我们的方法的准确度提升了 8.75%。此外，我们的方法在中文分类法数据集上的表现优于 ChatGPT。