We introduce MuseChat, an innovative dialog-based music recommendation
system. This unique platform not only offers interactive user engagement but
also suggests music tailored for input videos, so that users can refine and
personalize their music selections. In contrast, previous systems predominantly
emphasized content compatibility, often overlooking the nuances of users'
individual preferences. For example, all the datasets only provide basic
music-video pairings or such pairings with textual music descriptions. To
address this gap, our research offers three contributions. First, we devise a
conversation-synthesis method that simulates a two-turn interaction between a
user and a recommendation system, which leverages pre-trained music tags and
artist information. In this interaction, users submit a video to the system,
which then suggests a suitable music piece with a rationale. Afterwards, users
communicate their musical preferences, and the system presents a refined music
recommendation with reasoning. Second, we introduce a multi-modal
recommendation engine that matches music either by aligning it with visual cues
from the video or by harmonizing visual information, feedback from previously
recommended music, and the user's textual input. Third, we bridge music
representations and textual data with a Large Language Model(Vicuna-7B). This
alignment equips MuseChat to deliver music recommendations and their underlying
reasoning in a manner resembling human communication. Our evaluations show that
MuseChat surpasses existing state-of-the-art models in music retrieval tasks
and pioneers the integration of the recommendation process within a natural
language framework.

MuseChat 是一个创新的对话式音乐推荐系统，通过模拟用户与推荐系统之间的互动，利用预训练的音乐标签和艺术家信息，将音乐推荐过程融合到自然语言框架中，以人类交流方式提供音乐推荐和推荐原因，以图像和文本搭配的多模态推荐引擎，使音乐匹配音视频中的视觉线索或协调视觉信息、之前推荐音乐的反馈和用户的文本输入，MuseChat 在音乐检索任务中超过现有的最先进模型，是推荐过程与自然语言框架相结合的先驱。

MuseChat: 视频的对话式音乐推荐系统

MuseChat: A Conversational Music Recommendation System for Videos

A Music Recommendation System based on Emotion, Age, and Ethnicity is
developed in this study, using FER-2013 and ``Age, Gender, and Ethnicity (Face
Data) CSV'' datasets. The CNN architecture, which is extensively used for this
kind of purpose has been applied to the training of the models. After adding
several appropriate layers to the training end of the project, in total, 3
separate models are trained in the Deep Learning side of the project: Emotion,
Ethnicity, and Age. After the training step of these models, they are used as
classifiers on the web application side. The snapshot of the user taken through
the interface is sent to the models to predict their mood, age, and ethnic
origin. According to these classifiers, various kinds of playlists pulled from
Spotify API are proposed to the user in order to establish a functional and
user-friendly atmosphere for the music selection. Afterward, the user can
choose the playlist they want and listen to it by following the given link.

使用 FER-2013 和 “年龄、性别和种族（面部数据）CSV” 数据集，基于情感、年龄和种族开发了一种音乐推荐系统，并使用 CNN 架构对模型进行了训练，训练了 3 个不同的模型，用于为用户推荐具有功能性和用户友好的音乐播放列表。

基于情感、年龄和种族的音乐推荐系统

Music Recommendation System based on Emotion, Age and Ethnicity

Advanced music recommendation systems are being introduced along with the
development of machine learning. However, it is essential to design a music
recommendation system that can increase user satisfaction by understanding
users' music tastes, not by the complexity of models. Although several studies
related to music recommendation systems exploiting negative preferences have
shown performance improvements, there was a lack of explanation on how they led
to better recommendations. In this work, we analyze the role of negative
preference in users' music tastes by comparing music recommendation models with
contrastive learning exploiting preference (CLEP) but with three different
training strategies - exploiting preferences of both positive and negative
(CLEP-PN), positive only (CLEP-P), and negative only (CLEP-N). We evaluate the
effectiveness of the negative preference by validating each system with a small
amount of personalized data obtained via survey and further illuminate the
possibility of exploiting negative preference in music recommendations. Our
experimental results show that CLEP-N outperforms the other two in accuracy and
false positive rate. Furthermore, the proposed training strategies produced a
consistent tendency regardless of different types of front-end musical feature
extractors, proving the stability of the proposed method.

本研究采用对比学习 (contrastive learning) 模型并结合三种不同的训练策略来分析用户对负向偏好在音乐口味中的作用，证实了将负向偏好纳入音乐推荐系统的可行性，并且实验证明了负向偏好训练策略对于提高音乐推荐准确度和减少误报率的有效性。