Multi-task learning commonly encounters competition for resources among
tasks, specifically when model capacity is limited. This challenge motivates
models which allow control over the relative importance of tasks and total
compute cost during inference time. In this work, we propose such a
controllable multi-task network that dynamically adjusts its architecture and
weights to match the desired task preference as well as the resource
constraints. In contrast to the existing dynamic multi-task approaches that
adjust only the weights within a fixed architecture, our approach affords the
flexibility to dynamically control the total computational cost and match the
user-preferred task importance better. We propose a disentangled training of
two hypernetworks, by exploiting task affinity and a novel branching
regularized loss, to take input preferences and accordingly predict
tree-structured models with adapted weights. Experiments on three multi-task
benchmarks, namely PASCAL-Context, NYU-v2, and CIFAR-100, show the efficacy of
our approach. Project page is available at this https URL

本文提出了一种可控的多任务网络，该网络根据用户所需的任务偏好和资源限制以动态方式调整其架构和权重，并通过利用任务亲和性和一种新的分支规则化损失来训练两个超级网络以预测适应权重的树状模型。