Differentiable optimization has received a significant amount of attention due to its foundational role in the domain of machine learning based on neural networks. The existing methods leverages the optimality conditions and implicit function theorem to obtain the Jacobian matrix of the output, which increases the computational cost and limits the application of differentiable optimization. In addition, some non-differentiable constraints lead to more challenges when using prior differentiable optimization layers. This paper proposes a differentiable layer, named Differentiable Frank-Wolfe Layer (DFWLayer), by rolling out the Frank-Wolfe method, a well-known optimization algorithm which can solve constrained optimization problems without projections and Hessian matrix computations, thus leading to a efficient way of dealing with large-scale problems. Theoretically, we establish a bound on the suboptimality gap of the DFWLayer in the context of l1-norm constraints. Experimental assessments demonstrate that the DFWLayer not only attains competitive accuracy in solutions and gradients but also consistently adheres to constraints. Moreover, it surpasses the baselines in both forward and backward computational speeds.

该论文介绍了一种名为Differentiable Frank-Wolfe Layer（DFWLayer）的可微分层，借助Frank-Wolfe方法解决了带约束优化问题，从而提供了处理大规模问题的高效方法。实验证明DFWLayer在解决方案和梯度方面具有竞争性的准确性，并始终遵守约束条件，在前向和反向计算速度方面超越了基线。

可微分的Frank-Wolfe优化层