For long-tailed classification, most works often pretrain a big model on a
large-scale dataset, and then fine-tune the whole model for adapting to
long-tailed data. Though promising, fine-tuning the whole pretrained model
tends to suffer from high cost in computation and deployment of different
models for different tasks, as well as weakened generalization ability for
overfitting to certain features of long-tailed data. To alleviate these issues,
we propose an effective Long-tailed Prompt Tuning method for long-tailed
classification. LPT introduces several trainable prompts into a frozen
pretrained model to adapt it to long-tailed data. For better effectiveness, we
divide prompts into two groups: 1) a shared prompt for the whole long-tailed
dataset to learn general features and to adapt a pretrained model into target
domain; and 2) group-specific prompts to gather group-specific features for the
samples which have similar features and also to empower the pretrained model
with discrimination ability. Then we design a two-phase training paradigm to
learn these prompts. In phase 1, we train the shared prompt via supervised
prompt tuning to adapt a pretrained model to the desired long-tailed domain. In
phase 2, we use the learnt shared prompt as query to select a small best
matched set for a group of similar samples from the group-specific prompt set
to dig the common features of these similar samples, then optimize these
prompts with dual sampling strategy and asymmetric GCL loss. By only
fine-tuning a few prompts while fixing the pretrained model, LPT can reduce
training and deployment cost by storing a few prompts, and enjoys a strong
generalization ability of the pretrained model. Experiments show that on
various long-tailed benchmarks, with only ~1.1% extra parameters, LPT achieves
comparable performance than previous whole model fine-tuning methods, and is
more robust to domain-shift.

提出了一种有效的长尾 Prompt 调整方法用于长尾分类。该方法使用两个阶段的培训范例来学习训练可调节的提示符。实验证明，与之前的整个模型微调方法相比，此方法具有可比较的性能，并且更加强大。