The entropy error function has been widely used in neural networks.
Nevertheless, the network training based on this error function generally leads
to a slow convergence rate, and can easily be trapped in a local minimum or
even with the incorrect saturation problem in practice. In fact, there are many
results based on entropy error function in neural network and its applications.
However, the theory of such an algorithm and its convergence have not been
fully studied so far. To tackle the issue, we propose a novel entropy function
with smoothing l0 regularization for feed-forward neural networks. Using
real-world datasets, we performed an empirical evaluation to demonstrate that
the newly conceived algorithm allows us to substantially improve the prediction
performance of the considered neural networks. More importantly, the
experimental results also show that our proposed function brings in more
precise classifications, compared to well-founded baselines. Our work is novel
as it enables neural networks to learn effectively, producing more accurate
predictions compared to state-of-the-art algorithms. In this respect, we expect
that the algorithm will contribute to existing studies in the field, advancing
research in Machine Learning and Deep Learning.

基于熵误差函数的神经网络收敛速度慢，容易被困在局部最小值或错误的饱和问题中。为了解决这个问题，我们提出了一种新的带有平滑 l0 正则化的熵函数，用于前馈神经网络，并通过实证评估表明该算法能够大幅提高神经网络的预测性能，并且相比于基线模型，我们的算法可以产生更精确的分类结果。

用平滑 l0 正则化的熵误差函数的 SGD 方法用于神经网络

SGD method for entropy error function with smoothing l0 regularization  for neural networks

We examined the use of the Ising model as an L0 regularization method for
field-aware factorization machines (FFM). This approach improves generalization
performance and has the advantage of simultaneously determining the best
feature combinations for each of several groups. We can deepen the
interpretation and understanding of the model from the similarities and
differences in the features selected in each group.

使用 Ising 模型作为 L0 正则化方法提高场感知分解机 (FFM) 的泛化性能，同时确定每个组的最佳特征组合，从每个组选择的特征的相似性和差异性进一步解释和理解模型。

通过 Ising 模型对领域感知因子分解机进行 L0 正则化

L0 Regularization of Field-Aware Factorization Machine through Ising  Model

The alternating direction method of multipliers (ADMM) is a common
optimization tool for solving constrained and non-differentiable problems. We
provide an empirical study of the practical performance of ADMM on several
nonconvex applications, including l0 regularized linear regression, l0
regularized image denoising, phase retrieval, and eigenvector computation. Our
experiments suggest that ADMM performs well on a broad class of non-convex
problems. Moreover, recently proposed adaptive ADMM methods, which
automatically tune penalty parameters as the method runs, can improve algorithm
efficiency and solution quality compared to ADMM with a non-tuned penalty.

本文为非凸优化问题提供一种广泛适用的解决方法，即交替方向乘子法（ADMM），并研究了其在不同领域的实用性，包括线性回归、图像降噪、相位恢复及特征向量计算， 最近从提出的自适应 ADMM 方法可以通过自动调整惩罚参数来提高算法效率和解决质量 compared to ADMM with a non-tuned penalty。