In enhancing LinkedIn core content recommendation models, a significant
challenge lies in improving their semantic understanding capabilities. This
paper addresses the problem by leveraging multi-task learning, a method that
has shown promise in various domains. We fine-tune a pre-trained,
transformer-based LLM using multi-task contrastive learning with data from a
diverse set of semantic labeling tasks. We observe positive transfer, leading
to superior performance across all tasks when compared to training
independently on each. Our model outperforms the baseline on zero shot learning
and offers improved multilingual support, highlighting its potential for
broader application. The specialized content embeddings produced by our model
outperform generalized embeddings offered by OpenAI on Linkedin dataset and
tasks. This work provides a robust foundation for vertical teams across
LinkedIn to customize and fine-tune the LLM to their specific applications. Our
work offers insights and best practices for the field to build on.

通过利用多任务学习来改进 LinkedIn 核心内容推荐模型的语义理解能力，本研究提出了一种方法。我们使用来自不同语义标注任务的数据，通过多任务对比学习，对预训练的基于 Transformer 的 LLM 进行微调。我们观察到正向转移，相比于独立训练每个任务，在所有任务上都表现出更好的性能。我们的模型在零 - shot 学习上优于基准并提供了改进的多语言支持，突显了其广泛应用的潜力。我们模型产生的专门内容嵌入优于 OpenAI 在 Linkedin 数据集和任务上提供的通用嵌入。该工作为 LinkedIn 的垂直团队提供了一个健壮的基础，可以根据他们的特定应用定制和微调 LLM。我们的工作为该领域提供了见解和最佳实践。