Pre-trained Language Models (PLMs) are known to contain various kinds of
knowledge. One method to infer relational knowledge is through the use of
cloze-style prompts, where a model is tasked to predict missing subjects or
objects. Typically, designing these prompts is a tedious task because small
differences in syntax or semantics can have a substantial impact on knowledge
retrieval performance. Simultaneously, evaluating the impact of either prompt
syntax or information is challenging due to their interdependence. We designed
CONPARE-LAMA - a dedicated probe, consisting of 34 million distinct prompts
that facilitate comparison across minimal paraphrases. These paraphrases follow
a unified meta-template enabling the controlled variation of syntax and
semantics across arbitrary relations. CONPARE-LAMA enables insights into the
independent impact of either syntactical form or semantic information of
paraphrases on the knowledge retrieval performance of PLMs. Extensive knowledge
retrieval experiments using our probe reveal that prompts following clausal
syntax have several desirable properties in comparison to appositive syntax: i)
they are more useful when querying PLMs with a combination of supplementary
information, ii) knowledge is more consistently recalled across different
combinations of supplementary information, and iii) they decrease response
uncertainty when retrieving known facts. In addition, range information can
boost knowledge retrieval performance more than domain information, even though
domain information is more reliably helpful across syntactic forms.

预训练语言模型可以通过闭式样式提示来推断关系性知识，使用统一的元模板设计的 CONPARE-LAMA 探针表明句子结构在知识检索性能方面具有多种可取的特性，并且领域信息相对于语法形式更可靠地提升知识检索性能。