The inductive biases of trained neural networks are difficult to understand and, consequently, to adapt to new settings. We study the inductive biases of linearizations of neural networks, which we show to be surprisingly good summaries of the full network functions. Inspired by this finding, we propose a technique for embedding these inductive biases into Gaussian processes through a kernel designed from the Jacobian of the network. In this setting, domain adaptation takes the form of interpretable posterior inference, with accompanying uncertainty estimation. This inference is analytic and free of local optima issues found in standard techniques such as fine-tuning neural network weights to a new task. We develop significant computational speed-ups based on matrix multiplies, including a novel implementation for scalable Fisher vector products. Our experiments on both image classification and regression demonstrate the promise and convenience of this framework for transfer learning, compared to neural network fine-tuning. Code is available at https://github.com/amzn/xfer/tree/master/finite_ntk.

本文研究神经网络的归纳偏差，探讨了线性化的神经网络函数作为完整神经网络函数的概括之间的关系，并提出了一种新的方法将这种归纳偏差嵌入到高斯过程中以实现可解释的后验推断，无需采用标准技术（如微调神经网络权重）进行领域自适应。实验证明，该框架对于迁移学习非常有前途和便利。

线性化神经网络下的快速适应