Learning dynamics is at the heart of many important applications of machine learning (ML), such as robotics and autonomous driving. In these settings, ML algorithms typically need to reason about a physical system using high dimensional observations, such as images, without access to the underlying state. Recently, several methods have proposed to integrate priors from classical mechanics into ML models to address the challenge of physical reasoning from images. In this work, we take a sober look at the current capabilities of these models. To this end, we introduce a suite consisting of 17 datasets with visual observations based on physical systems exhibiting a wide range of dynamics. We conduct a thorough and detailed comparison of the major classes of physically inspired methods alongside several strong baselines. While models that incorporate physical priors can often learn latent spaces with desirable properties, our results demonstrate that these methods fail to significantly improve upon standard techniques. Nonetheless, we find that the use of continuous and time-reversible dynamics benefits models of all classes.

本文研究如何通过强化机器学习模型的物理先验知识来解决物理图像理解问题，提出了一种包含 17 个物理系统的数据集，对当前物理启发式机器学习方法进行了细致的对比分析，结果表明尽管这些方法通常可以学习到具有良好性质的隐藏空间，但没有显著提升标准技术的性能。

哪些先验知识是重要的？学习潜在动态的模型基准测试