Tuning scientific and probabilistic machine learning models -- for example,
partial differential equations, Gaussian processes, or Bayesian neural networks
-- often relies on evaluating functions of matrices whose size grows with the
data set or the number of parameters. While the state-of-the-art for evaluating
these quantities is almost always based on Lanczos and Arnoldi iterations, the
present work is the first to explain how to differentiate these workhorses of
numerical linear algebra efficiently. To get there, we derive previously
unknown adjoint systems for Lanczos and Arnoldi iterations, implement them in
JAX, and show that the resulting code can compete with Diffrax when it comes to
differentiating PDEs, GPyTorch for selecting Gaussian process models and beats
standard factorisation methods for calibrating Bayesian neural networks. All
this is achieved without any problem-specific code optimisation. Find the code
at this https URL and install the
library with pip install matfree.

通过导出 Lanczos 和 Arnoldi 迭代的未知共轭系统，该研究利用 JAX 实现了用于科学机器学习模型的数值线性代数效率高的可微分算法，该算法成功地不需要特定问题的代码优化，并在不同领域如选取高斯过程模型和校准贝叶斯神经网络中表现出与其他常见方法相当甚至更好的效果。