We propose a categorical semantics for machine learning algorithms in terms
of lenses, parametric maps, and reverse derivative categories. This foundation
provides a powerful explanatory and unifying framework: it encompasses a
variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov
momentum, as well as a variety of loss functions such as MSE and Softmax
cross-entropy, and different architectures, shedding new light on their
similarities and differences. Furthermore, our approach to learning has
examples generalising beyond the familiar continuous domains (modelled in
categories of smooth maps) and can be realised in the discrete setting of
Boolean and polynomial circuits. We demonstrate the practical significance of
our framework with an implementation in Python.

我们提出了一种基于镜头、参数化映射和反向导数范畴的机器学习算法的分类语义学。这一基础提供了一个强大的解释性和统一性框架：它包括了各种梯度下降算法，如 ADAM、AdaGrad 和 Nesterov 动量，以及各种损失函数，如 MSE 和 Softmax 交叉熵，以及不同的架构，揭示了它们的相似性和差异。此外，我们的学习方法超越了熟悉的连续领域的示例（在光滑映射范畴中建模），并且可以应用于布尔电路和多项式电路的离散环境。我们通过 Python 实现展示了我们框架的实际意义。

参数化透镜的深度学习

Deep Learning with Parametric Lenses

We propose a categorical semantics of gradient-based machine learning
algorithms in terms of lenses, parametrised maps, and reverse derivative
categories. This foundation provides a powerful explanatory and unifying
framework: it encompasses a variety of gradient descent algorithms such as
ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions
such as as MSE and Softmax cross-entropy, shedding new light on their
similarities and differences. Our approach to gradient-based learning has
examples generalising beyond the familiar continuous domains (modelled in
categories of smooth maps) and can be realized in the discrete setting of
boolean circuits. Finally, we demonstrate the practical significance of our
framework with an implementation in Python.

本文提出了一种基于 lens、参数化映射和反向导数类别的梯度机器学习算法的范畴语义学方法。该方法不只包含了 ADAM、AdaGrad 和 Nesterov momentum 等多种梯度下降算法，还将 MSE 和 Softmax 交叉熵等多种损失函数包含在内，从而阐明它们的相似性和差异性。同时，该方法不仅适用于连续域，还可以推广到布尔电路的离散设定。此外，文中还以 Python 语言实现了该方法的具体应用。

基于梯度的学习的范畴基础

Categorical Foundations of Gradient-Based Learning

While unsupervised domain translation (UDT) has seen a lot of success
recently, we argue that mediating its translation via categorical semantic
features could broaden its applicability. In particular, we demonstrate that
categorical semantics improves the translation between perceptually different
domains sharing multiple object categories. We propose a method to learn, in an
unsupervised manner, categorical semantic features (such as object labels) that
are invariant of the source and target domains. We show that conditioning the
style encoder of unsupervised domain translation methods on the learned
categorical semantics leads to a translation preserving the digits on
MNIST$\leftrightarrow$SVHN and to a more realistic stylization on
Sketches$\to$Reals.

本文提出，利用分类语义特征对无监督领域翻译进行干预可以拓宽其适用性，通过学习语义特征（如对象标签）的方法，可以更好地保持数字的传输和能够实现更逼真的描绘。