Levin et al. (2019) show experimentally that the interpretations of novel
English noun compounds (e.g., stew skillet), while not fully compositional, are
highly predictable based on whether the modifier and head refer to artifacts or
natural kinds. Is the large language model GPT-3 governed by the same
interpretive principles? To address this question, we first compare Levin et
al.'s experimental data with GPT-3 generations, finding a high degree of
similarity. However, this evidence is consistent with GPT3 reasoning only about
specific lexical items rather than the more abstract conceptual categories of
Levin et al.'s theory. To probe more deeply, we construct prompts that require
the relevant kind of conceptual reasoning. Here, we fail to find convincing
evidence that GPT-3 is reasoning about more than just individual lexical items.
These results highlight the importance of controlling for low-level
distributional regularities when assessing whether a large language model
latently encodes a deeper theory.

研究 GPT-3 对英语名词复合词的解释是否遵循同样的概念原则，通过实验和构建提示，没有找到令人信服的证据来证明 GPT-3 的解释不仅限于单个词汇项目。