Despite a lack of theoretical understanding, deep neural networks have
achieved unparalleled performance in a wide range of applications. On the other
hand, shallow representation learning with component analysis is associated
with rich intuition and theory, but smaller capacity often limits its
usefulness. To bridge this gap, we introduce Deep Component Analysis (DeepCA),
an expressive multilayer model formulation that enforces hierarchical structure
through constraints on latent variables in each layer. For inference, we
propose a differentiable optimization algorithm implemented using recurrent
Alternating Direction Neural Networks (ADNNs) that enable parameter learning
using standard backpropagation. By interpreting feed-forward networks as
single-iteration approximations of inference in our model, we provide both a
novel theoretical perspective for understanding them and a practical technique
for constraining predictions with prior knowledge. Experimentally, we
demonstrate performance improvements on a variety of tasks, including
single-image depth prediction with sparse output constraints.

本文介绍了一种表达能力强且带有层次结构约束的 Deep Component Analysis 模型，并提出了基于递归交替方向神经网络（ADNNs）的可微优化算法实现。通过将前馈神经网络解释为模型中推理的单次迭代近似，不仅提供了一种新的理论视角，还提供了一种利用先验知识约束预测的实用技术。在实验中，我们证明了该模型在许多任务上的性能改进，包括单图像深度预测与稀疏输出约束。