Scientific computing is increasingly incorporating the advancements in
machine learning and the ability to work with large amounts of data. At the
same time, machine learning models are becoming increasingly sophisticated and
exhibit many features often seen in scientific computing, stressing the
capabilities of machine learning frameworks. Just as the disciplines of
scientific computing and machine learning have shared common underlying
infrastructure in the form of numerical linear algebra, we now have the
opportunity to further share new computational infrastructure, and thus ideas,
in the form of Differentiable Programming. We describe Zygote, a Differentiable
Programming system that is able to take gradients of general program
structures. We implement this system in the Julia programming language. Our
system supports almost all language constructs (control flow, recursion,
mutation, etc.) and compiles high-performance code without requiring any user
intervention or refactoring to stage computations. This enables an expressive
programming model for deep learning, but more importantly, it enables us to
incorporate a large ecosystem of libraries in our models in a straightforward
way. We discuss our approach to automatic differentiation, including its
support for advanced techniques such as mixed-mode, complex and checkpointed
differentiation, and present several examples of differentiating programs.

这篇论文介绍了一种 Differentiable Programming 系统 Zygote，能够从一般的程序结构中获取梯度，该系统支持控制流、递归、突变等几乎所有语言构造，并支持各种高级自动微分技术，可以在不需要任何用户干预或重构来分阶段执行计算的情况下编译高性能代码，从而使得我们能够在深度学习模型中简单地集成一个大的库生态系统。