Tasks in multi-task learning often correlate, conflict, or even compete with
each other. As a result, a single solution that is optimal for all tasks rarely
exists. Recent papers introduced the concept of Pareto optimality to this field
and directly cast multi-task learning as multi-objective optimization problems,
but solutions returned by existing methods are typically finite, sparse, and
discrete. We present a novel, efficient method that generates locally
continuous Pareto sets and Pareto fronts, which opens up the possibility of
continuous analysis of Pareto optimal solutions in machine learning problems.
We scale up theoretical results in multi-objective optimization to modern
machine learning problems by proposing a sample-based sparse linear system, for
which standard Hessian-free solvers in machine learning can be applied. We
compare our method to the state-of-the-art algorithms and demonstrate its usage
of analyzing local Pareto sets on various multi-task classification and
regression problems. The experimental results confirm that our algorithm
reveals the primary directions in local Pareto sets for trade-off balancing,
finds more solutions with different trade-offs efficiently, and scales well to
tasks with millions of parameters.

提出了一种新的、高效的方法，可以生成局部连续的 Pareto 集和 Pareto fronts，并将其应用于现代机器学习问题中。通过提出基于样本的稀疏线性系统，将多目标优化的理论结果扩展到现代机器学习问题中，并实现了局部 Pareto 集的分析。与现有算法相比，通过在各种多任务分类和回归问题上的应用，证明了我们的算法在平衡权衡、有效地找到更多的不同权衡解以及迎合百万级参数任务的能力。

多任务学习中高效的连续 Pareto 探索

Efficient Continuous Pareto Exploration in Multi-Task Learning

The community detection problem for graphs asks one to partition the n
vertices V of a graph G into k communities, or clusters, such that there are
many intracluster edges and few intercluster edges. Of course this is
equivalent to finding a permutation matrix P such that, if A denotes the
adjacency matrix of G, then PAP^T is approximately block diagonal. As there are
k^n possible partitions of n vertices into k subsets, directly determining the
optimal clustering is clearly infeasible. Instead one seeks to solve a more
tractable approximation to the clustering problem. In this paper we reformulate
the community detection problem via sparse solution of a linear system
associated with the Laplacian of a graph G and then develop a two-stage
approach based on a thresholding technique and a compressive sensing algorithm
to find a sparse solution which corresponds to the community containing a
vertex of interest in G. Crucially, our approach results in an algorithm which
is able to find a single cluster of size n_0 in O(nlog(n)n_0) operations and
all k clusters in fewer than O(n^2ln(n)) operations. This is a marked
improvement over the classic spectral clustering algorithm, which is unable to
find a single cluster at a time and takes approximately O(n^3) operations to
find all k clusters. Moreover, we are able to provide robust guarantees of
success for the case where G is drawn at random from the Stochastic Block
Model, a popular model for graphs with clusters. Extensive numerical results
are also provided, showing the efficacy of our algorithm on both synthetic and
real-world data sets.

该研究提出了基于稀疏线性系统和两阶段阈值和压缩感知算法的社区检测问题解决方案，在较小计算量内有效地在 Stochastic Block Model 模型的随机图数据以及真实数据集上实现图的划分。