Meta-learning has emerged as an effective methodology to model several
real-world tasks and problems due to its extraordinary effectiveness in the
low-data regime. There are many scenarios ranging from the classification of
rare diseases to language modelling of uncommon languages where the
availability of large datasets is rare. Similarly, for more broader scenarios
like self-driving, an autonomous vehicle needs to be trained to handle every
situation well. This requires training the ML model on a variety of tasks with
good quality data. But often times, we find that the data distribution across
various tasks is skewed, i.e.the data follows a long-tail distribution. This
leads to the model performing well on some tasks and not performing so well on
others leading to model robustness issues. Meta-learning has recently emerged
as a potential learning paradigm which can effectively learn from one task and
generalize that learning to unseen tasks. In this study, we aim to exploit
external knowledge of task relations to improve training stability via
effective mini-batching of tasks. We hypothesize that selecting a diverse set
of tasks in a mini-batch will lead to a better estimate of the full gradient
and hence will lead to a reduction of noise in training.

通过选择多样的任务进行小批量处理，我们研究利用任务关联的外部知识来改善训练稳定性，进而减少训练中的噪音。

探索任务内部关系以改善元学习算法

Exploring intra-task relations to improve meta-learning algorithms

Autonomous robots are increasingly utilized in realistic scenarios with
multiple complex tasks. In these scenarios, there may be a preferred way of
completing all of the given tasks, but it is often in conflict with optimal
execution. Recent work studies preference-based planning, however, they have
yet to extend the notion of preference to the behavior of the robot with
respect to each task. In this work, we introduce a novel notion of preference
that provides a generalized framework to express preferences over individual
tasks as well as their relations. Then, we perform an optimal trade-off
(Pareto) analysis between behaviors that adhere to the user's preference and
the ones that are resource optimal. We introduce an efficient planning
framework that generates Pareto-optimal plans given user's preference by
extending A* search. Further, we show a method of computing the entire Pareto
front (the set of all optimal trade-offs) via an adaptation of a
multi-objective A* algorithm. We also present a problem-agnostic search
heuristic to enable scalability. We illustrate the power of the framework on
both mobile robots and manipulators. Our benchmarks show the effectiveness of
the heuristic with up to 2-orders of magnitude speedup.

本文提出一种新颖的优先级概念，可表达对每个任务及其关系的偏好，并通过对 A * 搜索的扩展来生成符合用户偏好和资源优化的帕累托最优计划。同时，通过多目标 A * 算法适应性改进计算所有最优的取舍，并提出一个问题无关搜索启发式以实现可扩展性，在移动机器人和机器人操纵者上展示框架的效力，达到了 2 个数量级的加速。

多个时间任务下的最优成本偏好权衡规划

Optimal Cost-Preference Trade-off Planning with Multiple Temporal Tasks

Multitask learning is a framework that enforces multiple learning tasks to
share knowledge to improve their generalization abilities. While shallow
multitask learning can learn task relations, it can only handle predefined
features. Modern deep multitask learning can jointly learn latent features and
task sharing, but they are obscure in task relation. Also, they predefine which
layers and neurons should share across tasks and cannot learn adaptively. To
address these challenges, this paper proposes a new multitask learning
framework that jointly learns latent features and explicit task relations by
complementing the strength of existing shallow and deep multitask learning
scenarios. Specifically, we propose to model the task relation as the
similarity between task input gradients, with a theoretical analysis of their
equivalency. In addition, we innovatively propose a multitask learning
objective that explicitly learns task relations by a new regularizer.
Theoretical analysis shows that the generalizability error has been reduced
thanks to the proposed regularizer. Extensive experiments on several multitask
learning and image classification benchmarks demonstrate the proposed method
effectiveness, efficiency as well as reasonableness in the learned task
relation patterns.

该论文提出了一种新的多任务学习框架，通过补充现有的浅层和深度多任务学习场景的优势，联合学习潜在特征和显式任务关系。同时，通过新的正则化器显式地学习任务关系。理论分析表明，该方法降低了泛化误差，实验表明该方法是有效的、高效的并且合理的。

显著性正则化深度多任务学习

Saliency-Regularized Deep Multi-Task Learning

Multi-task learning (MTL) in deep neural networks for NLP has recently
received increasing interest due to some compelling benefits, including its
potential to efficiently regularize models and to reduce the need for labeled
data. While it has brought significant improvements in a number of NLP tasks,
mixed results have been reported, and little is known about the conditions
under which MTL leads to gains in NLP. This paper sheds light on the specific
task relations that can lead to gains from MTL models over single-task setups.

本文阐述了多任务深度神经网络在 NLP 领域中的应用和优势，并探究了其在不同任务相关性下提高 NLP 性能的条件和作用。