In this paper, we address the issue of modeling and estimating changes in the
state of the spatio-temporal dynamical systems based on a sequence of
observations like video frames. Traditional numerical simulation systems depend
largely on the initial settings and correctness of the constructed partial
differential equations (PDEs). Despite recent efforts yielding significant
success in discovering data-driven PDEs with neural networks, the limitations
posed by singular scenarios and the absence of local insights prevent them from
performing effectively in a broader real-world context. To this end, this paper
propose the universal expert module -- that is, optical flow estimation
component, to capture the evolution laws of general physical processes in a
data-driven fashion. To enhance local insight, we painstakingly design a
finer-grained physical pipeline, since local characteristics may be influenced
by various internal contextual information, which may contradict the
macroscopic properties of the whole system. Further, we harness currently
popular neural discrete learning to unveil the underlying important features in
its latent space, this process better injects interpretability, which can help
us obtain a powerful prior over these discrete random variables. We conduct
extensive experiments and ablations to demonstrate that the proposed framework
achieves large performance margins, compared with the existing SOTA baselines.

本文提出了一种基于数据驱动的普适专家模块，即光流估计组件，用于捕捉广泛的实际物理过程的演化规律；通过精细的物理流程设计和神经离散学习，增强了局部洞察力并获得潜在空间中的重要特征。实验结果表明，与现有的基线方法相比，所提出的框架取得了显著的性能提升。