A method is presented that allows to reduce a problem described by
differential equations with initial and boundary conditions to the problem
described only by differential equations. The advantage of using the modified
problem for physics-informed neural networks (PINNs) methodology is that it
becomes possible to represent the loss function in the form of a single term
associated with differential equations, thus eliminating the need to tune the
scaling coefficients for the terms related to boundary and initial conditions.
The weighted loss functions respecting causality were modified and new weighted
loss functions based on generalized functions are derived. Numerical
experiments have been carried out for a number of problems, demonstrating the
accuracy of the proposed methods.

本研究提出了一种方法，可以将带有初始和边界条件的微分方程问题简化为仅由微分方程描述的问题，并基于广义函数构造了新的加权损失函数，以提高物理启发神经网络的准确性。

关于在尊重因果关系下训练物理知识驱动神经网络的最优损失函数

About optimal loss function for training physics-informed neural  networks under respecting causality

There are a plethora of methods and algorithms that solve the classical
multi-label document classification. However, when it comes to deployment and
usage in an industry setting, most, if not all the contemporary approaches fail
to address some of the vital aspects or requirements of an ideal solution: i.
ability to operate on variable-length texts and rambling documents. ii.
catastrophic forgetting problem. iii. modularity when it comes to online
learning and updating the model. iv. ability to spotlight relevant text while
producing the prediction, i.e. visualizing the predictions. v. ability to
operate on imbalanced or skewed datasets. vi. scalability. The paper describes
the significance of these problems in detail and proposes a unique neural
network architecture that addresses the above problems. The proposed
architecture views documents as a sequence of sentences and leverages
sentence-level embeddings for input representation. A hydranet-like
architecture is designed to have granular control over and improve the
modularity, coupled with a weighted loss driving task-specific heads. In
particular, two specific mechanisms are compared: Bi-LSTM and
Transformer-based. The architecture is benchmarked on some of the popular
benchmarking datasets such as Web of Science - 5763, Web of Science - 11967,
BBC Sports, and BBC News datasets. The experimental results reveal that the
proposed model outperforms the existing methods by a substantial margin. The
ablation study includes comparisons of the impact of the attention mechanism
and the application of weighted loss functions to train the task-specific heads
in the hydranet.

论文描述了经典的多标签文档分类的不足之处，提出了一种解决方案：通过基于句子级别嵌入的神经网络架构，实现能够处理可变长度文本和杂乱文档，具有模块化、可视化、操作不平衡数据和可扩展性等特点，并通过基准测试数据集 Web of Science、BBC Sports 和 BBC News 数据集的实验结果表明，该模型比现有方法优劣明显，加权损失函数和注意力机制对于训练具体任务有重要影响。