In this paper, we introduce a novel theoretical framework for multi-task
regression, applying random matrix theory to provide precise performance
estimations, under high-dimensional, non-Gaussian data distributions. We
formulate a multi-task optimization problem as a regularization technique to
enable single-task models to leverage multi-task learning information. We
derive a closed-form solution for multi-task optimization in the context of
linear models. Our analysis provides valuable insights by linking the
multi-task learning performance to various model statistics such as raw data
covariances, signal-generating hyperplanes, noise levels, as well as the size
and number of datasets. We finally propose a consistent estimation of training
and testing errors, thereby offering a robust foundation for hyperparameter
optimization in multi-task regression scenarios. Experimental validations on
both synthetic and real-world datasets in regression and multivariate time
series forecasting demonstrate improvements on univariate models, incorporating
our method into the training loss and thus leveraging multivariate information.

我们提出了一个新的理论框架，将随机矩阵理论应用于高维非高斯数据分布下，为多任务回归提供准确的性能估计；我们通过将多任务优化问题形式化为一种正则化技术来使单任务模型利用多任务学习信息，并在线性模型的背景下导出了多任务优化的闭式解；我们的分析通过将多任务学习性能与各种模型统计量（如原始数据协方差、信号生成超平面、噪声水平以及数据集的大小和数量）进行链接，提供了有价值的见解；最后，我们提出了对训练和测试误差的一致估计，从而为多任务回归场景中的超参数优化提供了稳健的基础；将我们的方法与训练损失相结合，从而利用多变量信息，实验证实了在回归和多变量时间序列预测的合成和真实世界数据集上对于单变量模型的改进。