Representation learning plays a crucial role in automated feature selection,
particularly in the context of high-dimensional data, where non-parametric
methods often struggle. In this study, we focus on supervised learning
scenarios where the pertinent information resides within a lower-dimensional
linear subspace of the data, namely the multi-index model. If this subspace
were known, it would greatly enhance prediction, computation, and
interpretation. To address this challenge, we propose a novel method for linear
feature learning with non-parametric prediction, which simultaneously estimates
the prediction function and the linear subspace. Our approach employs empirical
risk minimisation, augmented with a penalty on function derivatives, ensuring
versatility. Leveraging the orthogonality and rotation invariance properties of
Hermite polynomials, we introduce our estimator, named RegFeaL. By utilising
alternative minimisation, we iteratively rotate the data to improve alignment
with leading directions and accurately estimate the relevant dimension in
practical settings. We establish that our method yields a consistent estimator
of the prediction function with explicit rates. Additionally, we provide
empirical results demonstrating the performance of RegFeaL in various
experiments.

利用非参数预测的线性特征学习方法 RegFeaL，同时估计预测函数和线性子空间，经过实验证实其在不同实验中的表现。

非参数线性特征学习与回归正则化

Nonparametric Linear Feature Learning in Regression Through  Regularisation

We discuss a general method to learn data representations from multiple
tasks. We provide a justification for this method in both settings of multitask
learning and learning-to-learn. The method is illustrated in detail in the
special case of linear feature learning. Conditions on the theoretical
advantage offered by multitask representation learning over independent task
learning are established. In particular, focusing on the important example of
half-space learning, we derive the regime in which multitask representation
learning is beneficial over independent task learning, as a function of the
sample size, the number of tasks and the intrinsic data dimensionality. Other
potential applications of our results include multitask feature learning in
reproducing kernel Hilbert spaces and multilayer, deep networks.

讨论了一般从多个任务中学习数据表示的方法，并在多任务学习和学习到学习的两种情况下对此方法进行了理论上的证明。我们通过线性特征学习的特例对方法进行详细说明，并建立起多任务表示学习在独立任务学习上的理论优势的条件，特别是针对重要的半空间学习的例子，我们推导出了多任务表示学习在独立任务学习上的优势随样本数量、任务数量和固有数据维度的函数关系，其他潜在应用包括在再现核希尔伯特空间和多层深度网络中进行多任务特征学习。