Soft-thresholding has been widely used in neural networks. Its basic network
structure is a two-layer convolution neural network with soft-thresholding. Due
to the network's nature of nonlinearity and nonconvexity, the training process
heavily depends on an appropriate initialization of network parameters,
resulting in the difficulty of obtaining a globally optimal solution. To
address this issue, a convex dual network is designed here. We theoretically
analyze the network convexity and numerically confirm that the strong duality
holds. This conclusion is further verified in the linear fitting and denoising
experiments. This work provides a new way to convexify soft-thresholding neural
networks.

本文提出一个新的凸化软阈值神经网络的方法，设计一个凸对偶网络来解决网络的非线性和非凸性造成的全局最优解难以获得的问题，并在线性拟合和降噪实验中验证了此结论。

软阈值下二层卷积神经网络的凸对偶理论分析

Convex Dual Theory Analysis of Two-Layer Convolutional Neural Networks  with Soft-Thresholding

This paper investigates in which cases continuous optimization for directed
acyclic graph (DAG) structure learning can and cannot perform well and why this
happens, and suggests possible directions to make the search procedure more
reliable. Reisach et al. (2021) suggested that the remarkable performance of
several continuous structure learning approaches is primarily driven by a high
agreement between the order of increasing marginal variances and the
topological order, and demonstrated that these approaches do not perform well
after data standardization. We analyze this phenomenon for continuous
approaches assuming equal and non-equal noise variances, and show that the
statement may not hold in either case by providing counterexamples,
justifications, and possible alternative explanations. We further demonstrate
that nonconvexity may be a main concern especially for the non-equal noise
variances formulation, while recent advances in continuous structure learning
fail to achieve improvement in this case. Our findings suggest that future
works should take into account the non-equal noise variances formulation to
handle more general settings and for a more comprehensive empirical evaluation.
Lastly, we provide insights into other aspects of the search procedure,
including thresholding and sparsity, and show that they play an important role
in the final solutions.

本研究探讨了连续优化在有向无环图结构学习中表现良好和表现不佳的情况及原因，并提供了可能的改进方向。研究发现，非等噪声方差情况下存在非凸性问题，而连续结构学习的最新进展未能在此情况下实现改进。因此，未来研究应考虑非等噪声方差以处理更广泛的设置并进行更全面的实证评估。

连续优化结构学习：审慎地观察和开拓

Structure Learning with Continuous Optimization: A Sober Look and Beyond

As science and engineering have become increasingly data-driven, the role of
optimization has expanded to touch almost every stage of the data analysis
pipeline, from signal and data acquisition to modeling and prediction. The
optimization problems encountered in practice are often nonconvex. While
challenges vary from problem to problem, one common source of nonconvexity is
nonlinearity in the data or measurement model. Nonlinear models often exhibit
symmetries, creating complicated, nonconvex objective landscapes, with multiple
equivalent solutions. Nevertheless, simple methods (e.g., gradient descent)
often perform surprisingly well in practice.
The goal of this survey is to highlight a class of tractable nonconvex
problems, which can be understood through the lens of symmetries. These
problems exhibit a characteristic geometric structure: local minimizers are
symmetric copies of a single "ground truth" solution, while other critical
points occur at balanced superpositions of symmetric copies of the ground
truth, and exhibit negative curvature in directions that break the symmetry.
This structure enables efficient methods to obtain global minimizers. We
discuss examples of this phenomenon arising from a wide range of problems in
imaging, signal processing, and data analysis. We highlight the key role of
symmetry in shaping the objective landscape and discuss the different roles of
rotational and discrete symmetries. This area is rich with observed phenomena
and open problems; we close by highlighting directions for future research.

本文探讨了研究数据驱动最优化问题时遇到的非凸问题，这些问题可以通过对称性的角度来理解它们表现出的几何结构及其与目标函数构建的相关性，并讨论了未来研究的方向。

从对称性到几何性：易处理的非凸问题

From Symmetry to Geometry: Tractable Nonconvex Problems

This paper will serve as an introduction to the body of work on robust
subspace recovery. Robust subspace recovery involves finding an underlying
low-dimensional subspace in a dataset that is possibly corrupted with outliers.
While this problem is easy to state, it has been difficult to develop optimal
algorithms due to its underlying nonconvexity. This work emphasizes advantages
and disadvantages of proposed approaches and unsolved problems in the area.

此论文为关于鲁棒性子空间恢复的工作作为介绍。该工作重点介绍了现有方法的优缺点以及该领域中未解决的问题，其目标是在可能被离群值破坏的数据集中查找潜在的低维子空间。