Predicting multiple heterogeneous biological and medical targets is a
challenge for traditional deep learning models. In contrast to single-task
learning, in which a separate model is trained for each target, multi-task
learning (MTL) optimizes a single model to predict multiple related targets
simultaneously. To address this challenge, we propose the Multi-gate
Mixture-of-Experts with Exclusivity (MMoEEx). Our work aims to tackle the
heterogeneous MTL setting, in which the same model optimizes multiple tasks
with different characteristics. Such a scenario can overwhelm current MTL
approaches due to the challenges in balancing shared and task-specific
representations and the need to optimize tasks with competing optimization
paths. Our method makes two key contributions: first, we introduce an approach
to induce more diversity among experts, thus creating representations more
suitable for highly imbalanced and heterogenous MTL learning; second, we adopt
a two-step optimization [6, 11] approach to balancing the tasks at the gradient
level. We validate our method on three MTL benchmark datasets, including
Medical Information Mart for Intensive Care (MIMIC-III) and PubChem BioAssay
(PCBA).

提出了一种名为 MMoEEx 的多门分层专家模型用于解决多任务学习（MTL）中的挑战，旨在优化单个模型以同时预测具有不同特征的多个任务，我们采用了一种方法来引入专家间的多样性，以创造更适合高度不平衡和异构 MTL 学习的表示，并采用两步优化方法在梯度水平上平衡任务。