Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCMC)
algorithm that generates proposals for a Metropolis-Hastings algorithm by
simulating the dynamics of a Hamiltonian system. However, HMC is sensitive to
large time discretizations and performs poorly if there is a mismatch between
the spatial geometry of the target distribution and the scales of the momentum
distribution. In particular the mass matrix of HMC is hard to tune well. In
order to alleviate these problems we propose relativistic Hamiltonian Monte
Carlo, a version of HMC based on relativistic dynamics that introduce a maximum
velocity on particles. We also derive stochastic gradient versions of the
algorithm and show that the resulting algorithms bear interesting relationships
to gradient clipping, RMSprop, Adagrad and Adam, popular optimisation methods
in deep learning. Based on this, we develop relativistic stochastic gradient
descent by taking the zero-temperature limit of relativistic stochastic
gradient Hamiltonian Monte Carlo. In experiments we show that the relativistic
algorithms perform better than classical Newtonian variants and Adam.

提出了基于相对论动力学的哈密顿蒙特卡罗方法，通过引入粒子的最大速度解决哈密顿蒙特卡罗在大时间离散化和空间几何不匹配时的性能问题，并开发了基于此的相对论随机梯度下降算法，与深度学习中的优化方法如梯度截断、RMSprop、Adagrad 和 Adam 有趣的关系，实验表明这种算法比经典牛顿变体和 Adam 表现更好。