Recent developments in large-scale machine learning models for
general-purpose understanding, translation and generation of language are
driving impact across a variety of sectors including medicine, robotics, and
scientific discovery. The strength of such Large Language Models (LLMs) stems
from the large corpora that they are trained with. While this imbues them with
a breadth of capabilities, they have been found unsuitable for some specific
types of problems such as advanced mathematics. In this paper, we highlight the
inability of LLMs to reason about physics tasks. We demonstrate that their
ability to infer parameters of physical systems can be improved, without
retraining, by augmenting their context with feedback from physical simulation.

大规模机器学习模型在语言的通用理解、翻译和生成方面的最新进展，给医学、机器人学和科学探索等多个领域带来了影响。然而，这些大语言模型因为其训练所使用的大型语料库的特点，存在无法处理一些特定问题，如高级数学。本文针对大语言模型无法进行物理任务推理的问题进行了研究，并证明了通过从物理模拟的反馈中增强它们的上下文，可以改进它们推断物理系统参数的能力，而无需重新训练。

SimLM：语言模型能否推断物理系统的参数？

SimLM: Can Language Models Infer Parameters of Physical Systems?

Graph-based next-step prediction models have recently been very successful in
modeling complex high-dimensional physical systems on irregular meshes.
However, due to their short temporal attention span, these models suffer from
error accumulation and drift. In this paper, we propose a new method that
captures long-term dependencies through a transformer-style temporal attention
model. We introduce an encoder-decoder structure to summarize features and
create a compact mesh representation of the system state, to allow the temporal
model to operate on a low-dimensional mesh representations in a memory
efficient manner. Our method outperforms a competitive GNN baseline on several
complex fluid dynamics prediction tasks, from sonic shocks to vascular flow. We
demonstrate stable rollouts without the need for training noise and show
perfectly phase-stable predictions even for very long sequences. More broadly,
we believe our approach paves the way to bringing the benefits of
attention-based sequence models to solving high-dimensional complex physics
tasks.

本文提出一种基于图的下一步预测模型，采用了一种类 Transformer 的时序关注模型来捕捉长期的依赖关系，利用编码器 - 解码器结构来总结特征和创建系统状态的紧凑网格表示，并在几个复杂的流体力学预测任务中优于竞争性 GNN 基线，可以帮助解决高维物理问题的序列建模。