Recent studies on pre-trained vision/language models have demonstrated the
practical benefit of a new, promising solution-building paradigm in AI where
models can be pre-trained on broad data describing a generic task space and
then adapted successfully to solve a wide range of downstream tasks, even when
training data is severely limited (e.g., in zero- or few-shot learning
scenarios). Inspired by such progress, we investigate in this paper the
possibilities and challenges of adapting such a paradigm to the context of
recommender systems, which is less investigated from the perspective of
pre-trained model. In particular, we propose to develop a generic recommender
that captures universal interaction patterns by training on generic user-item
interaction data extracted from different domains, which can then be fast
adapted to improve few-shot learning performance in unseen new domains (with
limited data).
However, unlike vision/language data which share strong conformity in the
semantic space, universal patterns underlying recommendation data collected
across different domains (e.g., different countries or different E-commerce
platforms) are often occluded by both in-domain and cross-domain biases
implicitly imposed by the cultural differences in their user and item bases, as
well as their uses of different e-commerce platforms. As shown in our
experiments, such heterogeneous biases in the data tend to hinder the
effectiveness of the pre-trained model. To address this challenge, we further
introduce and formalize a causal debiasing perspective, which is substantiated
via a hierarchical Bayesian deep learning model, named PreRec. Our empirical
studies on real-world data show that the proposed model could significantly
improve the recommendation performance in zero- and few-shot learning settings
under both cross-market and cross-platform scenarios.

通过训练泛域用户 - 物品交互数据，我们提出了一个适应性泛型推荐系统，该系统能够捕捉通用交互模式，并能在不同领域中快速适应以提高零样本和少样本学习性能。然而，不同领域的推荐数据存在域内和跨域偏差，我们通过引入因果去偏视角和预测模型 PreRec 有效克服了这一挑战，实证结果表明在跨市场和跨平台场景下，该模型能显著提升推荐性能。

预训练推荐系统：因果去偏见观点

Pre-trained Recommender Systems: A Causal Debiasing Perspective

Many studies focus on improving pretraining or developing new backbones in
text-video retrieval. However, existing methods may suffer from the learning
and inference bias issue, as recent research suggests in other
text-video-related tasks. For instance, spatial appearance features on action
recognition or temporal object co-occurrences on video scene graph generation
could induce spurious correlations. In this work, we present a unique and
systematic study of a temporal bias due to frame length discrepancy between
training and test sets of trimmed video clips, which is the first such attempt
for a text-video retrieval task, to the best of our knowledge. We first
hypothesise and verify the bias on how it would affect the model illustrated
with a baseline study. Then, we propose a causal debiasing approach and perform
extensive experiments and ablation studies on the Epic-Kitchens-100, YouCook2,
and MSR-VTT datasets. Our model overpasses the baseline and SOTA on nDCG, a
semantic-relevancy-focused evaluation metric which proves the bias is
mitigated, as well as on the other conventional metrics.

改进和开发新的网络架构是文本 - 视频检索的研究热点。然而，现有的方法可能存在学习和推理偏差问题，本研究首次尝试探索修剪视频剪辑的训练和测试集之间的时间偏差，并提出因果去偏方法，通过大量实验验证了该方法的有效性。