Language models~(LMs) gradually become general-purpose interfaces in the
interactive and embodied world, where the understanding of physical concepts is
an essential prerequisite. However, it is not yet clear whether LMs can
understand physical concepts in the human world. To investigate this, we design
a benchmark VEC that covers the tasks of (i) Visual concepts, such as the shape
and material of objects, and (ii) Embodied Concepts, learned from the
interaction with the world such as the temperature of objects. Our zero
(few)-shot prompting results show that the understanding of certain visual
concepts emerges as scaling up LMs, but there are still basic concepts to which
the scaling law does not apply. For example, OPT-175B performs close to humans
with a zero-shot accuracy of 85\% on the material concept, yet behaves like
random guessing on the mass concept. Instead, vision-augmented LMs such as CLIP
and BLIP achieve a human-level understanding of embodied concepts. Analysis
indicates that the rich semantics in visual representation can serve as a
valuable source of embodied knowledge. Inspired by this, we propose a
distillation method to transfer embodied knowledge from VLMs to LMs, achieving
performance gain comparable with that by scaling up the parameters of LMs 134x.
Our dataset is available at https://github.com/TobiasLee/VEC

该论文研究了自然语言模型是否能理解物理概念，并设计了一个基准测试 VEC 来探究视觉和具身概念。结果表明，视觉增强的语言模型，如 CLIP 和 BLIP，能够理解具身概念，而通过蒸馏方法将具身知识转移给 LMs 的性能也相当可观。

语言模型能够理解物理概念吗？

Can Language Models Understand Physical Concepts?

Figurative language is a challenge for language models since its
interpretation is based on the use of words in a way that deviates from their
conventional order and meaning. Yet, humans can easily understand and interpret
metaphors, similes or idioms as they can be derived from embodied metaphors.
Language is a proxy for embodiment and if a metaphor is conventional and
lexicalised, it becomes easier for a system without a body to make sense of
embodied concepts. Yet, the intricate relation between embodiment and features
such as concreteness or age of acquisition has not been studied in the context
of figurative language interpretation concerning language models. Hence, the
presented study shows how larger language models perform better at interpreting
metaphoric sentences when the action of the metaphorical sentence is more
embodied. The analysis rules out multicollinearity with other features (e.g.
word length or concreteness) and provides initial evidence that larger language
models conceptualise embodied concepts to a degree that facilitates figurative
language understanding.

本研究采用大型语言模型，研究对比了具有体现概念的隐喻句子和不具有的，结果表明具有体现概念的隐喻句子更易被解释。