Classic algorithms and machine learning systems like neural networks are both
abundant in everyday life. While classic computer science algorithms are
suitable for precise execution of exactly defined tasks such as finding the
shortest path in a large graph, neural networks allow learning from data to
predict the most likely answer in more complex tasks such as image
classification, which cannot be reduced to an exact algorithm. To get the best
of both worlds, this thesis explores combining both concepts leading to more
robust, better performing, more interpretable, more computationally efficient,
and more data efficient architectures. The thesis formalizes the idea of
algorithmic supervision, which allows a neural network to learn from or in
conjunction with an algorithm. When integrating an algorithm into a neural
architecture, it is important that the algorithm is differentiable such that
the architecture can be trained end-to-end and gradients can be propagated back
through the algorithm in a meaningful way. To make algorithms differentiable,
this thesis proposes a general method for continuously relaxing algorithms by
perturbing variables and approximating the expectation value in closed form,
i.e., without sampling. In addition, this thesis proposes differentiable
algorithms, such as differentiable sorting networks, differentiable renderers,
and differentiable logic gate networks. Finally, this thesis presents
alternative training strategies for learning with algorithms.

本论文探讨了算法监督的概念，提出了一种连续松弛算法的一般方法，定量评估了算法和神经网络的组合对性能和效率的增益，并探讨了在算法中引入可微性的各种不同技术和训练策略。