Can computers perceive the physical properties of objects solely through vision? Research in cognitive science and vision science has shown that humans excel at identifying materials and estimating their physical properties based purely on visual appearance. In this paper, we present a novel approach for dense prediction of the physical properties of objects using a collection of images. Inspired by how humans reason about physics through vision, we leverage large language models to propose candidate materials for each object. We then construct a language-embedded point cloud and estimate the physical properties of each 3D point using a zero-shot kernel regression approach. Our method is accurate, annotation-free, and applicable to any object in the open world. Experiments demonstrate the effectiveness of the proposed approach in various physical property reasoning tasks, such as estimating the mass of common objects, as well as other properties like friction and hardness.

计算机通过视觉能否感知物体的物理属性？我们提出了一种利用图像集合对物体的物理属性进行密集预测的新方法，通过借鉴人类如何通过视觉进行物理推理的方式，利用大型语言模型为每个物体提出候选材料，然后构建一个嵌入语言的点云，并利用零样本核回归方法估计每个三维点的物理属性，我们的方法准确、无需注释，并适用于开放世界中的任何对象。实验证明了该方法在各种物理属性推理任务中的有效性，如估计常见物体的质量以及摩擦和硬度等其他属性。

基于语言嵌入特征场的物性理解