The development of large vision-language models (LVLMs) offers the potential
to address challenges faced by traditional multimodal recommendations thanks to
their proficient understanding of static images and textual dynamics. However,
the application of LVLMs in this field is still limited due to the following
complexities: First, LVLMs lack user preference knowledge as they are trained
from vast general datasets. Second, LVLMs suffer setbacks in addressing
multiple image dynamics in scenarios involving discrete, noisy, and redundant
image sequences. To overcome these issues, we propose the novel reasoning
scheme named Rec-GPT4V: Visual-Summary Thought (VST) of leveraging large
vision-language models for multimodal recommendation. We utilize user history
as in-context user preferences to address the first challenge. Next, we prompt
LVLMs to generate item image summaries and utilize image comprehension in
natural language space combined with item titles to query the user preferences
over candidate items. We conduct comprehensive experiments across four datasets
with three LVLMs: GPT4-V, LLaVa-7b, and LLaVa-13b. The numerical results
indicate the efficacy of VST.

利用大型视觉 - 语言模型进行多模态推荐的研究中，研究人员通过引入用户历史和基于图像摘要生成的查询方式，克服了大型视觉 - 语言模型面临的用户偏好知识和多样复杂图像序列问题，并验证了该方法的有效性。

Rec-GPT4V：大型视觉语言模型的多模态推荐

Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models

Click-Through Rate (CTR) prediction on cold users is a challenging task in
recommender systems. Recent researches have resorted to meta-learning to tackle
the cold-user challenge, which either perform few-shot user representation
learning or adopt optimization-based meta-learning. However, existing methods
suffer from information loss or inefficient optimization process, and they fail
to explicitly model global user preference knowledge which is crucial to
complement the sparse and insufficient preference information of cold users. In
this paper, we propose a novel and efficient approach named RESUS, which
decouples the learning of global preference knowledge contributed by collective
users from the learning of residual preferences for individual users.
Specifically, we employ a shared predictor to infer basis user preferences,
which acquires global preference knowledge from the interactions of different
users. Meanwhile, we develop two efficient algorithms based on the nearest
neighbor and ridge regression predictors, which infer residual user preferences
via learning quickly from a few user-specific interactions. Extensive
experiments on three public datasets demonstrate that our RESUS approach is
efficient and effective in improving CTR prediction accuracy on cold users,
compared with various state-of-the-art methods.

该论文提出了一种名为 RESUS 的新方法，通过学习全局用户首选项和个人用户残留偏好，从而有效提高了在冷用户上的点击率（CTR）预测精度。