The integration of constrained optimization models as components in deep
networks has led to promising advances on many specialized learning tasks. A
central challenge in this setting is backpropagation through the solution of an
optimization problem, which often lacks a closed form. One typical strategy is
algorithm unrolling, which relies on automatic differentiation through the
entire chain of operations executed by an iterative optimization solver. This
paper provides theoretical insights into the backward pass of unrolled
optimization, showing that it is asymptotically equivalent to the solution of a
linear system by a particular iterative method. Several practical pitfalls of
unrolling are demonstrated in light of these insights, and a system called
Folded Optimization is proposed to construct more efficient backpropagation
rules from unrolled solver implementations. Experiments over various end-to-end
optimization and learning tasks demonstrate the advantages of this system both
computationally, and in terms of flexibility over various optimization problem
forms.

通过算法展开的反向传播与通过迭代方法求解线性系统的解是渐进等价的，在构建更高效的反向传播规则方面提出了折叠优化系统。