Deep neural network architectures have recently produced excellent results in
a variety of areas in artificial intelligence and visual recognition, well
surpassing traditional shallow architectures trained using hand-designed
features. The power of deep networks stems both from their ability to perform
local computations followed by pointwise non-linearities over increasingly
larger receptive fields, and from the simplicity and scalability of the
gradient-descent training procedure based on backpropagation. An open problem
is the inclusion of layers that perform global, structured matrix computations
like segmentation (e.g. normalized cuts) or higher-order pooling (e.g.
log-tangent space metrics defined over the manifold of symmetric positive
definite matrices) while preserving the validity and efficiency of an
end-to-end deep training framework. In this paper we propose a sound
mathematical apparatus to formally integrate global structured computation into
deep computation architectures. At the heart of our methodology is the
development of the theory and practice of backpropagation that generalizes to
the calculus of adjoint matrix variations. The proposed matrix backpropagation
methodology applies broadly to a variety of problems in machine learning or
computational perception. Here we illustrate it by performing visual
segmentation experiments using the BSDS and MSCOCO benchmarks, where we show
that deep networks relying on second-order pooling and normalized cuts layers,
trained end-to-end using matrix backpropagation, outperform counterparts that
do not take advantage of such global layers.

本文提出了一种将全局结构化计算形式化整合到深度计算架构中的方法，其核心是开发了一种新的基于矩阵变化的 backpropagation 理论和实践，该方法广泛适用于机器学习或计算感知问题，我们在 BSDS 和 MSCOCO 基准测试中执行视觉分割实验，结果表明端到端训练的基于第二阶池化和归一化切割层的深度网络，使用矩阵反向传播优于没有利用全局层的对应网络。