We aim at exploiting additional auxiliary labels from an independent
(auxiliary) task to boost the primary task performance which we focus on, while
preserving a single task inference cost of the primary task. While most
existing auxiliary learning methods are optimization-based relying on loss
weights/gradients manipulation, our method is architecture-based with a
flexible asymmetric structure for the primary and auxiliary tasks, which
produces different networks for training and inference. Specifically, starting
from two single task networks/branches (each representing a task), we propose a
novel method with evolving networks where only primary-to-auxiliary links exist
as the cross-task connections after convergence. These connections can be
removed during the primary task inference, resulting in a single-task inference
cost. We achieve this by formulating a Neural Architecture Search (NAS)
problem, where we initialize bi-directional connections in the search space and
guide the NAS optimization converging to an architecture with only the
single-side primary-to-auxiliary connections. Moreover, our method can be
incorporated with optimization-based auxiliary learning approaches. Extensive
experiments with six tasks on NYU v2, CityScapes, and Taskonomy datasets using
VGG, ResNet, and ViT backbones validate the promising performance. The codes
are available at this https URL

利用神经架构搜索的方法，我们在主要任务的推理成本不变的前提下，通过利用独立辅助任务的额外辅助标签来提高主要任务的性能。这种方法使用柔性的非对称结构对主任务和辅助任务进行训练和推理，通过演化网络中的主到辅助连接，实现了在主任务推理过程中去除这些连接的单任务推理成本。我们的方法在 NYU v2、CityScapes 和 Taskonomy 数据集上使用 VGG、ResNet 和 ViT 等主干网络进行了广泛实验，验证了其有希望的性能。