Recently, using large pretrained Transformer models for transfer learning
tasks has evolved to the point where they have become one of the flagship
trends in the Natural Language Processing (NLP) community, giving rise to
various outlooks such as prompt-based, adapters or combinations with
unsupervised approaches, among many others. This work proposes a 3 Phase
technique to adjust a base model for a classification task. First, we adapt the
model's signal to the data distribution by performing further training with a
Denoising Autoencoder (DAE). Second, we adjust the representation space of the
output to the corresponding classes by clustering through a Contrastive
Learning (CL) method. In addition, we introduce a new data augmentation
approach for Supervised Contrastive Learning to correct the unbalanced
datasets. Third, we apply fine-tuning to delimit the predefined categories.
These different phases provide relevant and complementary knowledge to the
model to learn the final task. We supply extensive experimental results on
several datasets to demonstrate these claims. Moreover, we include an ablation
study and compare the proposed method against other ways of combining these
techniques.

使用大型预训练 Transformer 模型进行迁移学习任务已经成为自然语言处理（NLP）领域的主要趋势之一，本论文提出了一种三阶段技术来调整基础模型以适应分类任务。