This paper expands the damped block Newton (dBN) method introduced recently
in [4] for 1D diffusion-reaction equations and least-squares data fitting
problems. To determine the linear parameters (the weights and bias of the
output layer) of the neural network (NN), the dBN method requires solving
systems of linear equations involving the mass matrix. While the mass matrix
for local hat basis functions is tri-diagonal and well-conditioned, the mass
matrix for NNs is dense and ill-conditioned. For example, the condition number
of the NN mass matrix for quasi-uniform meshes is at least ${\cal O}(n^4)$. We
present a factorization of the mass matrix that enables solving the systems of
linear equations in ${\cal O}(n)$ operations. To determine the non-linear
parameters (the weights and bias of the hidden layer), one step of a damped
Newton method is employed at each iteration. A Gauss-Newton method is used in
place of Newton for the instances in which the Hessian matrices are singular.
This modified dBN is referred to as dBGN. For both methods, the computational
cost per iteration is ${\cal O}(n)$. Numerical results demonstrate the ability
dBN and dBGN to efficiently achieve accurate results and outperform BFGS for
select examples.

该研究论文介绍了一种扩展的阻尼块牛顿法，用于解决神经网络中包含质量矩阵的线性方程组，以及求解非线性参数的方法，该方法在计算代价上具有较高效性，并且优于 BFGS 算法。

神经网络方法的快速迭代求解器：II. 1D 扩散 - 反应问题与数据拟合

Fast Iterative Solver For Neural Network Method: II. 1D  Diffusion-Reaction Problems And Data Fitting

Hamiltonian Monte Carlo (HMC) is a popular Markov Chain Monte Carlo (MCMC)
algorithm to sample from an unnormalized probability distribution. A leapfrog
integrator is commonly used to implement HMC in practice, but its performance
can be sensitive to the choice of mass matrix used therein. We develop a
gradient-based algorithm that allows for the adaptation of the mass matrix by
encouraging the leapfrog integrator to have high acceptance rates while also
exploring all dimensions jointly. In contrast to previous work that adapt the
hyperparameters of HMC using some form of expected squared jumping distance,
the adaptation strategy suggested here aims to increase sampling efficiency by
maximizing an approximation of the proposal entropy. We illustrate that using
multiple gradients in the HMC proposal can be beneficial compared to a single
gradient-step in Metropolis-adjusted Langevin proposals. Empirical evidence
suggests that the adaptation method can outperform different versions of HMC
schemes by adjusting the mass matrix to the geometry of the target distribution
and by providing some control on the integration time.

本文提出了一种基于梯度的算法，允许通过鼓励 Leapfrog 积分器具有高接受率的方式来适应质量矩阵，最大化建议熵的近似值，实验证明该自适应方法可以通过调整质量矩阵来提高 HMC 的性能。