Recent developments in Large Language Models (LLMs) have demonstrated their
remarkable capabilities across a range of tasks. Questions, however, persist
about the nature of LLMs and their potential to integrate common-sense human
knowledge when performing tasks involving information about the real physical
world. This paper delves into these questions by exploring how LLMs can be
extended to interact with and reason about the physical world through IoT
sensors and actuators, a concept that we term "\textit{Penetrative AI}". The
paper explores such an extension at two levels of LLMs' ability to penetrate
into the physical world via the processing of sensory signals. Our preliminary
findings indicate that LLMs, with ChatGPT being the representative example in
our exploration, have considerable and unique proficiency in employing the
knowledge they learned during training for interpreting IoT sensor data and
reasoning over them about tasks in the physical realm. Not only this opens up
new applications for LLMs beyond traditional text-based tasks, but also enables
new ways of incorporating human knowledge in cyber-physical systems.

最近大型语言模型（LLMs）的发展展示了它们在各种任务中的显著能力。然而，关于 LLMs 的性质及其在涉及真实物理世界信息的任务中整合常识人类知识的潜力仍有疑问。本文通过探索 LLMs 如何通过物联网传感器和执行器与物理世界互动和推理，即所谓的 “穿透型人工智能”，深入探讨了这些问题。本文在两个层面上探索了 LLMs 渗透进物理世界的能力，通过处理感官信号。我们的初步发现表明，LLMs，尤其是 ChatGPT 作为我们探索的代表性例子，在利用它们在训练过程中学到的知识来解释物联网传感器数据并对其进行推理的任务中具有相当独特的能力和技巧。这不仅打开了 LLMs 在传统的基于文本任务之外的新应用领域，而且使得将人类知识纳入到网络物理系统中的方式得以实现。