In the field of multi-task reinforcement learning, the modular principle,
which involves specializing functionalities into different modules and
combining them appropriately, has been widely adopted as a promising approach
to prevent the negative transfer problem that performance degradation due to
conflicts between tasks. However, most of the existing multi-task RL methods
only combine shared modules at the task level, ignoring that there may be
conflicts within the task. In addition, these methods do not take into account
that without constraints, some modules may learn similar functions, resulting
in restricting the model's expressiveness and generalization capability of
modular methods. In this paper, we propose the Contrastive Modules with
Temporal Attention(CMTA) method to address these limitations. CMTA constrains
the modules to be different from each other by contrastive learning and
combining shared modules at a finer granularity than the task level with
temporal attention, alleviating the negative transfer within the task and
improving the generalization ability and the performance for multi-task RL. We
conducted the experiment on Meta-World, a multi-task RL benchmark containing
various robotics manipulation tasks. Experimental results show that CMTA
outperforms learning each task individually for the first time and achieves
substantial performance improvements over the baselines.

本文提出了一种称为对比模块与时间注意力（CMTA）的方法，通过对比学习同时结合细粒度的共享模块和时间注意力来解决现有多任务强化学习方法所存在的限制。CMTA 方法能够在任务内减少负迁移，提高模块化方法的泛化能力和性能，在 Meta-World 数据集上的实验结果表明 CMTA 优于单独学习每个任务，并在基准上取得了显著的性能改进。