While deep learning (DL) models are state-of-the-art in text and image
domains, they have not yet consistently outperformed Gradient Boosted Decision
Trees (GBDTs) on tabular Learning-To-Rank (LTR) problems. Most of the recent
performance gains attained by DL models in text and image tasks have used
unsupervised pretraining, which exploits orders of magnitude more unlabeled
data than labeled data. To the best of our knowledge, unsupervised pretraining
has not been applied to the LTR problem, which often produces vast amounts of
unlabeled data. In this work, we study whether unsupervised pretraining can
improve LTR performance over GBDTs and other non-pretrained models. Using
simple design choices--including SimCLR-Rank, our ranking-specific modification
of SimCLR (an unsupervised pretraining method for images)--we produce
pretrained deep learning models that soundly outperform GBDTs (and other
non-pretrained models) in the case where labeled data is vastly outnumbered by
unlabeled data. We also show that pretrained models also often achieve
significantly better robustness than non-pretrained models (GBDTs or DL models)
in ranking outlier data.

该论文研究了无监督预训练是否能提高在标注数据远远少于非标注数据情况下的 LTR 问题中的性能，并通过使用特定于排序的改进方法，在无监督预训练的深度学习模型中表现出比 GBDT 和其他非预训练模型更好的性能，并且在排名异常数据方面也通常具有显着更好的稳健性。