Structure learning of directed acyclic graphs (DAGs) is a fundamental problem in many scientific endeavors. A new line of work, based on NOTEARS (Zheng et al., 2018), reformulates the structure learning problem as a continuous optimization one by leveraging an algebraic characterization of DAG constraint. The constrained problem is typically solved using the augmented Lagrangian method (ALM) which is often preferred to the quadratic penalty method (QPM) by virtue of its convergence result that does not require the penalty coefficient to go to infinity, hence avoiding ill-conditioning. In this work, we review the standard convergence result of the ALM and show that the required conditions are not satisfied in the recent continuous constrained formulation for learning DAGs. We demonstrate empirically that its behavior is akin to that of the QPM which is prone to ill-conditioning, thus motivating the use of second-order method in this setting. We also establish the convergence guarantee of QPM to a DAG solution, under mild conditions, based on a property of the DAG constraint term.

该研究探讨了用增广Lagrange方法(ALM)和二次惩罚方法(QPM)求解结构学习的连续优化问题，发现ALM的收敛性质在线性、非线性和混淆情况下实际上和QPM相似，在QPM的渐近条件下收敛到有向无环图(DAG)解决方案，并将理论结果与现有方法相连接，验证了实验比较。

连续约束优化在结构学习中的收敛性