Multi-task reinforcement learning could enable robots to scale across a wide
variety of manipulation tasks in homes and workplaces. However, generalizing
from one task to another and mitigating negative task interference still
remains a challenge. Addressing this challenge by successfully sharing
information across tasks will depend on how well the structure underlying the
tasks is captured. In this work, we introduce our new architecture, Projected
Task-Specific Layers (PTSL), that leverages a common policy with dense
task-specific corrections through task-specific layers to better express shared
and variable task information. We then show that our model outperforms the
state of the art on the MT10 and MT50 benchmarks of Meta-World consisting of 10
and 50 goal-conditioned tasks for a Sawyer arm.

通过引入新的架构 Projected Task-Specific Layers (PTSL)，该论文通过使用特定任务层进行共享和可变任务信息的密集修正，从而成功地在多任务强化学习中解决了任务间的负面干扰，并在 Meta-World 的 MT10 和 MT50 基准测试中优于现有技术。

多任务强化学习的预测任务特定层

Projected Task-Specific Layers for Multi-Task Reinforcement Learning

A fundamental challenge for multi-task learning is that different tasks may
conflict with each other when they are solved jointly, and a cause of this
phenomenon is conflicting gradients during optimization. Recent works attempt
to mitigate the influence of conflicting gradients by directly altering the
gradients based on some criteria. However, our empirical study shows that
``gradient surgery'' cannot effectively reduce the occurrence of conflicting
gradients. In this paper, we take a different approach to reduce conflicting
gradients from the root. In essence, we investigate the task gradients w.r.t.
each shared network layer, select the layers with high conflict scores, and
turn them to task-specific layers. Our experiments show that such a simple
approach can greatly reduce the occurrence of conflicting gradients in the
remaining shared layers and achieve better performance, with only a slight
increase in model parameters in many cases. Our approach can be easily applied
to improve various state-of-the-art methods including gradient manipulation
methods and branched architecture search methods. Given a network architecture
(e.g., ResNet18), it only needs to search for the conflict layers once, and the
network can be modified to be used with different methods on the same or even
different datasets to gain performance improvement. The source code is
available at this https URL.

本文研究了多任务学习过程中由于梯度冲突导致的性能下降问题，提出了一种新的方法，在源头处减少了梯度冲突，并将高冲突得分的共享网络层替换为任务特定层，最终实现了更好的性能表现。

Recon: 从根节点减少多任务学习中的冲突梯度

Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learning

Real-world applications of object recognition often require the solution of
multiple tasks in a single platform. Under the standard paradigm of network
fine-tuning, an entirely new CNN is learned per task, and the final network
size is independent of task complexity. This is wasteful, since simple tasks
require smaller networks than more complex tasks, and limits the number of
tasks that can be solved simultaneously. To address these problems, we propose
a transfer learning procedure, denoted NetTailor, in which layers of a
pre-trained CNN are used as universal blocks that can be combined with small
task-specific layers to generate new networks. Besides minimizing
classification error, the new network is trained to mimic the internal
activations of a strong unconstrained CNN, and minimize its complexity by the
combination of 1) a soft-attention mechanism over blocks and 2) complexity
regularization constraints. In this way, NetTailor can adapt the network
architecture, not just its weights, to the target task. Experiments show that
networks adapted to simple tasks, such as character or traffic sign
recognition, become significantly smaller than those adapted to hard tasks,
such as fine-grained recognition. More importantly, due to the modular nature
of the procedure, this reduction in network complexity is achieved without
compromise of either parameter sharing across tasks, or classification
accuracy.

提出了一种基于迁移学习的网络微调方法 NetTailor，它使用预训练的 CNN 层作为通用模块，可以与小的任务特定层组合生成新的网络，通过软注意力机制和复杂度正则化约束，最小化分类错误的同时降低网络复杂度。该方法可以使得适应简单任务的网络结构比适应复杂任务的网络结构更小，并且不降低参数共享和分类精度。