Weblogs, comprised of records detailing user activities on any website, offer
valuable insights into user preferences, behavior, and interests. Numerous
recommendation algorithms, employing strategies such as collaborative
filtering, content-based filtering, and hybrid methods, leverage the data mined
through these weblogs to provide personalized recommendations to users. Despite
the abundance of information available in these weblogs, identifying and
extracting pertinent information and key features necessitates extensive
engineering endeavors. The intricate nature of the data also poses a challenge
for interpretation, especially for non-experts. In this study, we introduce a
sophisticated and interactive recommendation framework denoted as InteraRec,
which diverges from conventional approaches that exclusively depend on weblogs
for recommendation generation. This framework captures high-frequency
screenshots of web pages as users navigate through a website. Leveraging
state-of-the-art multimodal large language models (MLLMs), it extracts valuable
insights into user preferences from these screenshots by generating a user
behavioral summary based on predefined keywords. Subsequently, this summary is
utilized as input to an LLM-integrated optimization setup to generate tailored
recommendations. Through our experiments, we demonstrate the effectiveness of
InteraRec in providing users with valuable and personalized offerings.

通过捕获用户在网站上导航过程中的高频截图，利用多模态大型语言模型（MLLM）从这些截图中提取用户偏好的有价值洞察，基于预定义的关键词生成用户行为摘要，并将该摘要作为输入传递给 LLM 综合优化设置，从而产生量身定制的推荐，我们的实验证明了 InteraRec 在提供有价值和个性化的优惠方面的有效性。