Simultaneous feature selection and non-linear function estimation are
challenging, especially in high-dimensional settings where the number of
variables exceeds the available sample size in modeling. In this article, we
investigate the problem of feature selection in neural networks. Although the
group LASSO has been utilized to select variables for learning with neural
networks, it tends to select unimportant variables into the model to compensate
for its over-shrinkage. To overcome this limitation, we propose a framework of
sparse-input neural networks using group concave regularization for feature
selection in both low-dimensional and high-dimensional settings. The main idea
is to apply a proper concave penalty to the $l_2$ norm of weights from all
outgoing connections of each input node, and thus obtain a neural net that only
uses a small subset of the original variables. In addition, we develop an
effective algorithm based on backward path-wise optimization to yield stable
solution paths, in order to tackle the challenge of complex optimization
landscapes. Our extensive simulation studies and real data examples demonstrate
satisfactory finite sample performances of the proposed estimator, in feature
selection and prediction for modeling continuous, binary, and time-to-event
outcomes.

本文提出使用组凸正则化的稀疏输入神经网络框架，用于解决在高维数据中进行的变量筛选和非线性函数估计的问题。通过对每个输入节点的所有出节点的权重的 L2 范数应用适当的凹值惩罚，从而为每个问题生成一个小型子集，取得了令人满意的有限样本性能和稳定的解决方案。

使用组凸规则的稀疏输入神经网络

Sparse-Input Neural Network using Group Concave Regularization

We develop a penalized likelihood estimation framework to estimate the
structure of Gaussian Bayesian networks from observational data. In contrast to
recent methods which accelerate the learning problem by restricting the search
space, our main contribution is a fast algorithm for score-based structure
learning which does not restrict the search space in any way and works on
high-dimensional datasets with thousands of variables. Our use of concave
regularization, as opposed to the more popular $\ell_0$ (e.g. BIC) penalty, is
new. Moreover, we provide theoretical guarantees which generalize existing
asymptotic results when the underlying distribution is Gaussian. Most notably,
our framework does not require the existence of a so-called faithful DAG
representation, and as a result the theory must handle the inherent
nonidentifiability of the estimation problem in a novel way. Finally, as a
matter of independent interest, we provide a comprehensive comparison of our
approach to several standard structure learning methods using open-source
packages developed for the R language. Based on these experiments, we show that
our algorithm is significantly faster than other competing methods while
obtaining higher sensitivity with comparable false discovery rates for
high-dimensional data. In particular, the total runtime for our method to
generate a solution path of 20 estimates for DAGs with 8000 nodes is around one
hour.

本文提出了一种基于得分的结构学习快速算法的罚似然估计框架，该快速算法不受限制地处理高维数据集，并使用凹规则化以解决估计问题的固有不可识别性。其中，高斯贝叶斯网络是重点研究领域之一。

凹惩罚估计稀疏高斯贝叶斯网络

Concave Penalized Estimation of Sparse Gaussian Bayesian Networks

Concave regularization methods provide natural procedures for sparse
recovery. However, they are difficult to analyze in the high dimensional
setting. Only recently a few sparse recovery results have been established for
some specific local solutions obtained via specialized numerical procedures.
Still, the fundamental relationship between these solutions such as whether
they are identical or their relationship to the global minimizer of the
underlying nonconvex formulation is unknown. The current paper fills this
conceptual gap by presenting a general theoretical framework showing that under
appropriate conditions, the global solution of nonconvex regularization leads
to desirable recovery performance; moreover, under suitable conditions, the
global solution corresponds to the unique sparse local solution, which can be
obtained via different numerical procedures. Under this unified framework, we
present an overview of existing results and discuss their connections. The
unified view of this work leads to a more satisfactory treatment of concave
high dimensional sparse estimation procedures, and serves as guideline for
developing further numerical procedures for concave regularization.

本篇论文提出了一个理论框架，表明在适当的条件下，非凸正则化的全局解决方案可导致理想的稀疏恢复性能，并且在合适的条件下，全局解决方案对应于唯一的稀疏局部解决方案，其可通过不同的数值方法获得。该统一框架为凹形高维稀疏估计过程提供了更加令人满意的处理方法，并作为凹形正则化进一步数值程序发展的指导方针。