Multimedia recommendation involves personalized ranking tasks, where
multimedia content is usually represented using a generic encoder. However,
these generic representations introduce spurious correlations that fail to
reveal users' true preferences. Existing works attempt to alleviate this
problem by learning invariant representations, but overlook the balance between
independent and identically distributed (IID) and out-of-distribution (OOD)
generalization. In this paper, we propose a framework called Pareto Invariant
Representation Learning (PaInvRL) to mitigate the impact of spurious
correlations from an IID-OOD multi-objective optimization perspective, by
learning invariant representations (intrinsic factors that attract user
attention) and variant representations (other factors) simultaneously.
Specifically, PaInvRL includes three iteratively executed modules: (i)
heterogeneous identification module, which identifies the heterogeneous
environments to reflect distributional shifts for user-item interactions; (ii)
invariant mask generation module, which learns invariant masks based on the
Pareto-optimal solutions that minimize the adaptive weighted Invariant Risk
Minimization (IRM) and Empirical Risk (ERM) losses; (iii) convert module, which
generates both variant representations and item-invariant representations for
training a multi-modal recommendation model that mitigates spurious
correlations and balances the generalization performance within and cross the
environmental distributions. We compare the proposed PaInvRL with
state-of-the-art recommendation models on three public multimedia
recommendation datasets (Movielens, Tiktok, and Kwai), and the experimental
results validate the effectiveness of PaInvRL for both within- and
cross-environmental learning.

基于 Pareto 不变表示学习 (PaInvRL) 框架，同时学习不变表示和变体表示，减轻由通用编码器引入的错误相关性并提供良好的泛化性能。

多媒体推荐的帕累托不变表示学习

Pareto Invariant Representation Learning for Multimedia Recommendation

Multimedia content is of predominance in the modern Web era. In real
scenarios, multiple modalities reveal different aspects of item attributes and
usually possess different importance to user purchase decisions. However, it is
difficult for models to figure out users' true preference towards different
modalities since there exists strong statistical correlation between
modalities. Even worse, the strong statistical correlation might mislead models
to learn the spurious preference towards inconsequential modalities. As a
result, when data (modal features) distribution shifts, the learned spurious
preference might not guarantee to be as effective on the inference set as on
the training set. We propose a novel MOdality DEcorrelating STable learning
framework, MODEST for brevity, to learn users' stable preference. Inspired by
sample re-weighting techniques, the proposed method aims to estimate a weight
for each item, such that the features from different modalities in the weighted
distribution are decorrelated. We adopt Hilbert Schmidt Independence Criterion
(HSIC) as independence testing measure which is a kernel-based method capable
of evaluating the correlation degree between two multi-dimensional and
non-linear variables. Our method could be served as a play-and-plug module for
existing multimedia recommendation backbones. Extensive experiments on four
public datasets and four state-of-the-art multimedia recommendation backbones
unequivocally show that our proposed method can improve the performances by a
large margin.

该研究提出了一种 MODEST（MOdality DEcorrelating STable）学习框架，采用基于核的独立性检验测量方式，以学习用户对多媒体内容中不同模态的偏好，提高多媒体推荐性能。

挖掘稳定偏好：多媒体推荐的自适应模态解耦

Mining Stable Preferences: Adaptive Modality Decorrelation for  Multimedia Recommendation

With the prevalence of multimedia content on the Web, developing recommender
solutions that can effectively leverage the rich signal in multimedia data is
in urgent need. Owing to the success of deep neural networks in representation
learning, recent advance on multimedia recommendation has largely focused on
exploring deep learning methods to improve the recommendation accuracy. To
date, however, there has been little effort to investigate the robustness of
multimedia representation and its impact on the performance of multimedia
recommendation.
In this paper, we shed light on the robustness of multimedia recommender
system. Using the state-of-the-art recommendation framework and deep image
features, we demonstrate that the overall system is not robust, such that a
small (but purposeful) perturbation on the input image will severely decrease
the recommendation accuracy. This implies the possible weakness of multimedia
recommender system in predicting user preference, and more importantly, the
potential of improvement by enhancing its robustness. To this end, we propose a
novel solution named Adversarial Multimedia Recommendation (AMR), which can
lead to a more robust multimedia recommender model by using adversarial
learning. The idea is to train the model to defend an adversary, which adds
perturbations to the target image with the purpose of decreasing the model's
accuracy. We conduct experiments on two representative multimedia
recommendation tasks, namely, image recommendation and visually-aware product
recommendation. Extensive results verify the positive effect of adversarial
learning and demonstrate the effectiveness of our AMR method. Source codes are
available in this https URL

本文提出了 Adversarial Multimedia Recommendation (AMR) 方法，它使用对抗学习来提高多媒体推荐系统的鲁棒性。在对两个代表性的多媒体推荐任务进行实验后，结果验证了采用对抗学习的积极作用，并证明了我们的 AMR 方法的有效性。