While physics-informed neural networks (PINNs) have been proven effective for
low-dimensional partial differential equations (PDEs), the computational cost
remains a hurdle in high-dimensional scenarios. This is particularly pronounced
when computing high-order and high-dimensional derivatives in the
physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian
noise for stochastic smoothing of the original neural net model, enabling Monte
Carlo methods for derivative approximation, eliminating the need for costly
auto-differentiation. Despite its computational efficiency in high dimensions,
RS-PINN introduces biases in both loss and gradients, negatively impacting
convergence, especially when coupled with stochastic gradient descent (SGD). We
present a comprehensive analysis of biases in RS-PINN, attributing them to the
nonlinearity of the Mean Squared Error (MSE) loss and the PDE nonlinearity. We
propose tailored bias correction techniques based on the order of PDE
nonlinearity. The unbiased RS-PINN allows for a detailed examination of its
pros and cons compared to the biased version. Specifically, the biased version
has a lower variance and runs faster than the unbiased version, but it is less
accurate due to the bias. To optimize the bias-variance trade-off, we combine
the two approaches in a hybrid method that balances the rapid convergence of
the biased version with the high accuracy of the unbiased version. In addition,
we present an enhanced implementation of RS-PINN. Extensive experiments on
diverse high-dimensional PDEs, including Fokker-Planck, HJB, viscous Burgers',
Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off
and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are
provided for selecting biased, unbiased, or hybrid versions, depending on the
dimensionality and nonlinearity of the specific PDE problem.

通过引入高斯噪声进行随机平滑以及根据 PDE 非线性性提出的偏差校正技术，我们综合分析了 Randomized Smoothing PINN 的偏差问题，并在有限维度和非线性 PDE 问题上研究了 RS-PINN 的性能和效果。结果表明，在高维度问题中，将有偏差的版本与无偏差的版本相结合，可以在偏差和方差之间找到平衡点，以实现更快的收敛和更高的准确性。

具有随机平滑的物理信息神经网络中的偏差 - 方差权衡问题

Bias-Variance Trade-off in Physics-Informed Neural Networks with  Randomized Smoothing for High-Dimensional PDEs

Distance metric learning based on triplet loss has been applied with success
in a wide range of applications such as face recognition, image retrieval,
speaker change detection and recently recommendation with the CML model.
However, as we show in this article, CML requires large batches to work
reasonably well because of a too simplistic uniform negative sampling strategy
for selecting triplets. Due to memory limitations, this makes it difficult to
scale in high-dimensional scenarios. To alleviate this problem, we propose here
a 2-stage negative sampling strategy which finds triplets that are highly
informative for learning. Our strategy allows CML to work effectively in terms
of accuracy and popularity bias, even when the batch size is an order of
magnitude smaller than what would be needed with the default uniform sampling.
We demonstrate the suitability of the proposed strategy for recommendation and
exhibit consistent positive results across various datasets.

本文提出了一种针对距离度量学习的负采样策略，在推荐方面获得了不错的准确度和普及性偏差表现。该策略使得 CML 模型即使在批处理大小比默认采样策略需要的批处理大小小一个数量级的情况下，仍然能够高效地工作。