Large language models (LLMs) and their variants have shown extraordinary
efficacy across numerous downstream natural language processing (NLP) tasks,
which has presented a new vision for the development of NLP. Despite their
remarkable performance in natural language generating (NLG), LLMs lack a
distinct focus on the emotion understanding domain. As a result, using LLMs for
emotion recognition may lead to suboptimal and inadequate precision. Another
limitation of LLMs is that they are typical trained without leveraging
multi-modal information. To overcome these limitations, we propose DialogueLLM,
a context and emotion knowledge tuned LLM that is obtained by fine-tuning LLaMA
models with 13,638 multi-modal (i.e., texts and videos) emotional dialogues.
The visual information is considered as the supplementary knowledge to
construct high-quality instructions. We offer a comprehensive evaluation of our
proposed model on three benchmarking emotion recognition in conversations (ERC)
datasets and compare the results against the SOTA baselines and other SOTA
LLMs. Additionally, DialogueLLM-7B can be easily trained using LoRA on a 40GB
A100 GPU in 5 hours, facilitating reproducibility for other researchers.

使用多模态信息、上下文和情感知识调整的对话型大型语言模型 (DialogueLLM)，通过对 13,638 个多模态（文本和视频）情感对话进行微调，克服了大型语言模型在情感识别方面的局限性，并在三个基准情感对话识别数据集上进行了全面评估。