The problem of benign overfitting asks whether it is possible for a model to
perfectly fit noisy training data and still generalize well. We study benign
overfitting in two-layer leaky ReLU networks trained with the hinge loss on a
binary classification task. We consider input data which can be decomposed into
the sum of a common signal and a random noise component, which lie on subspaces
orthogonal to one another. We characterize conditions on the signal to noise
ratio (SNR) of the model parameters giving rise to benign versus non-benign, or
harmful, overfitting: in particular, if the SNR is high then benign overfitting
occurs, conversely if the SNR is low then harmful overfitting occurs. We
attribute both benign and non-benign overfitting to an approximate margin
maximization property and show that leaky ReLU networks trained on hinge loss
with Gradient Descent (GD) satisfy this property. In contrast to prior work we
do not require near orthogonality conditions on the training data: notably, for
input dimension $d$ and training sample size $n$, while prior work shows
asymptotically optimal error when $d = \Omega(n^2 \log n)$, here we require
only $d = \Omega\left(n \log \frac{1}{\epsilon}\right)$ to obtain error within
$\epsilon$ of optimal.

该研究针对二元分类任务，使用带有折线损失的两层泄漏整流线性单元网络，研究了良性过拟合问题，通过对模型参数的信号噪声比进行特征化，发现高信噪比出现良性过拟合，低信噪比出现有害过拟合，并将良性和非良性过拟合归因于近似边际最大化特性，同时降低了训练数据的正交性要求。

输入维度适中的带有泄漏 ReLU 的网络中的良性过拟合

Benign overfitting in leaky ReLU networks with moderate input dimension

Cancer detection is one of the key research topics in the medical field.
Accurate detection of different cancer types is valuable in providing better
treatment facilities and risk minimization for patients. This paper deals with
the classification problem of human cancer diseases by using gene expression
data. It is presented a new methodology to analyze microarray datasets and
efficiently classify cancer diseases. The new method first employs Signal to
Noise Ratio (SNR) to find a list of a small subset of non-redundant genes.
Then, after normalization, it is used Multi-Objective Particle Swarm
Optimization (MOPSO) for feature selection and employed Adaptive K-Nearest
Neighborhood (KNN) for cancer disease classification. This method improves the
classification accuracy of cancer classification by reducing the number of
features. The proposed methodology is evaluated by classifying cancer diseases
in five cancer datasets. The results are compared with the most recent
approaches, which increases the classification accuracy in each dataset.

本研究提出了一种新的方法，使用基因表达数据进行癌症分类问题的解决，首先运用信噪比算法筛选出非冗余基因的子集，然后运用多目标粒子群算法进行特征选择，并采用自适应 K 最近邻算法进行癌症分类，从而提高了癌症分类的准确性。

基于自适应 K 近邻的多目标 PSO 微阵列表达数据基因选择

Gene selection from microarray expression data: A Multi-objective PSO with adaptive K-nearest neighborhood

We consider the problem of learning a dictionary matrix from a number of
observed signals, which are assumed to be generated via a linear model with a
common underlying dictionary. In particular, we derive lower bounds on the
minimum achievable worst case mean squared error (MSE), regardless of
computational complexity of the dictionary learning (DL) schemes. By casting DL
as a classical (or frequentist) estimation problem, the lower bounds on the
worst case MSE are derived by following an established information-theoretic
approach to minimax estimation. The main conceptual contribution of this paper
is the adaption of the information-theoretic approach to minimax estimation for
the DL problem in order to derive lower bounds on the worst case MSE of any DL
scheme. We derive three different lower bounds applying to different generative
models for the observed signals. The first bound applies to a wide range of
models, it only requires the existence of a covariance matrix of the (unknown)
underlying coefficient vector. By specializing this bound to the case of sparse
coefficient distributions, and assuming the true dictionary satisfies the
restricted isometry property, we obtain a lower bound on the worst case MSE of
DL schemes in terms of a signal to noise ratio (SNR). The third bound applies
to a more restrictive subclass of coefficient distributions by requiring the
non-zero coefficients to be Gaussian. While, compared with the previous two
bounds, the applicability of this final bound is the most limited it is the
tightest of the three bounds in the low SNR regime.

本文论述了关于从多个观察值中学习字典矩阵的问题，并得出了三个不同的下限，分别应用于不同的生成模型，特别地，其中一下限是通过假设真实字典满足受限等同性性质，并基于信噪比来计算的。