Comparative knowledge (e.g., steel is stronger and heavier than styrofoam) is
an essential component of our world knowledge, yet understudied in prior
literature. In this paper, we study the task of comparative knowledge
acquisition, motivated by the dramatic improvements in the capabilities of
extreme-scale language models like GPT-3, which have fueled efforts towards
harvesting their knowledge into knowledge bases. However, access to inference
API for such models is limited, thereby restricting the scope and the diversity
of the knowledge acquisition. We thus ask a seemingly implausible question:
whether more accessible, yet considerably smaller and weaker models such as
GPT-2, can be utilized to acquire comparative knowledge, such that the
resulting quality is on par with their large-scale counterparts?
We introduce NeuroComparatives, a novel framework for comparative knowledge
distillation using lexically-constrained decoding, followed by stringent
filtering of generated knowledge. Our framework acquires comparative knowledge
between everyday objects and results in a corpus of 8.7M comparisons over 1.74M
entity pairs - 10X larger and 30% more diverse than existing resources.
Moreover, human evaluations show that NeuroComparatives outperform existing
resources (up to 32% absolute improvement), even including GPT-3, despite using
a 100X smaller model. Our results motivate neuro-symbolic manipulation of
smaller models as a cost-effective alternative to the currently dominant
practice of relying on extreme-scale language models with limited inference
access.

本文研究使用 GPT-2 等较小的语言模型获取比较知识的任务，提出一种新的比较知识提取框架 NeuroComparatives，并通过对所获得的 8.7M 个关系的人类验证，证明其优于现有资源（包括 GPT-3），这一结果表明适用于较小语言模型的神经符号操作是目前主流极大规模语言模型应用范围受限的一种有效的替代方案。