Task-conditional architecture offers advantage in parameter efficiency but
falls short in performance compared to state-of-the-art multi-decoder methods.
How to trade off performance and model parameters is an important and difficult
problem. In this paper, we introduce a simple and lightweight task-conditional
model called Prompt Guided Transformer (PGT) to optimize this challenge. Our
approach designs a Prompt-conditioned Transformer block, which incorporates
task-specific prompts in the self-attention mechanism to achieve global
dependency modeling and parameter-efficient feature adaptation across multiple
tasks. This block is integrated into both the shared encoder and decoder,
enhancing the capture of intra- and inter-task features. Moreover, we design a
lightweight decoder to further reduce parameter usage, which accounts for only
2.7% of the total model parameters. Extensive experiments on two multi-task
dense prediction benchmarks, PASCAL-Context and NYUD-v2, demonstrate that our
approach achieves state-of-the-art results among task-conditional methods while
using fewer parameters, and maintains a significant balance between performance
and parameter size.

本文介绍了一种简单且轻量级的任务条件模型，名为 Prompt Guided Transformer (PGT)，通过设计一个 Prompt-conditioned Transformer 块，在自注意机制中引入任务特定的提示，以实现全局依赖建模和跨多个任务的参数高效特征适应。实验证明，该方法在使用更少的参数的同时，取得了最先进的结果，并在性能和参数大小之间保持了显著的平衡。