We characterize and remedy a failure mode that may arise from multi-scale
dynamics with scale imbalances during training of deep neural networks, such as
Physics Informed Neural Networks (PINNs). PINNs are popular machine-learning
templates that allow for seamless integration of physical equation models with
data. Their training amounts to solving an optimization problem over a weighted
sum of data-fidelity and equation-fidelity objectives. Conflicts between
objectives can arise from scale imbalances, heteroscedasticity in the data,
stiffness of the physical equation, or from catastrophic interference during
sequential training. We explain the training pathology arising from this and
propose a simple yet effective inverse-Dirichlet weighting strategy to
alleviate the issue. We compare with Sobolev training of neural networks,
providing the baseline of analytically $\boldsymbol{\epsilon}$-optimal
training. We demonstrate the effectiveness of inverse-Dirichlet weighting in
various applications, including a multi-scale model of active turbulence, where
we show orders of magnitude improvement in accuracy and convergence over
conventional PINN training. For inverse modeling using sequential training, we
find that inverse-Dirichlet weighting protects a PINN against catastrophic
forgetting.

本文研究并解决了深度神经网络（如物理信息神经网络）训练过程中可能出现的多尺度动态和尺度失衡引发的故障模式，通过提出一种简单而有效的逆狄利克雷加权策略来缓解问题，并在各种应用中进行了验证和比较。

逆狄利克雷加权使物理信息神经网络训练可靠

Inverse-Dirichlet Weighting Enables Reliable Training of Physics  Informed Neural Networks

The evolution of social media popularity exhibits rich temporality, i.e.,
popularities change over time at various levels of temporal granularity. This
is influenced by temporal variations of public attentions or user activities.
For example, popularity patterns of street snap on Flickr are observed to
depict distinctive fashion styles at specific time scales, such as season-based
periodic fluctuations for Trench Coat or one-off peak in days for Evening
Dress. However, this fact is often overlooked by existing research of
popularity modeling. We present the first study to incorporate multiple
time-scale dynamics into predicting online popularity. We propose a novel
computational framework in the paper, named Multi-scale Temporalization, for
estimating popularity based on multi-scale decomposition and structural
reconstruction in a tensor space of user, post, and time by joint low-rank
constraints. By considering the noise caused by context inconsistency, we
design a data rearrangement step based on context aggregation as preprocessing
to enhance contextual relevance of neighboring data in the tensor space. As a
result, our approach can leverage multiple levels of temporal characteristics
and reduce the noise of data decomposition to improve modeling effectiveness.
We evaluate our approach on two large-scale Flickr image datasets with over 1.8
million photos in total, for the task of popularity prediction. The results
show that our approach significantly outperforms state-of-the-art popularity
prediction techniques, with a relative improvement of 10.9%-47.5% in terms of
prediction accuracy.

本文提出了一个名为 Multi-scale Temporalization 的新的计算框架，通过多尺度分解和结构重构来估计基于用户、帖子和时间的张量空间中的流行度，从而整合多个时间尺度动态来预测社交媒体影响力。通过在两个大规模 Flickr 图像数据集上进行实证评估，结果表明我们的方法在预测准确性方面具有明显的优势。