Multimodal intent recognition poses significant challenges, requiring the
incorporation of non-verbal modalities from real-world contexts to enhance the
comprehension of human intentions. Existing benchmark datasets are limited in
scale and suffer from difficulties in handling out-of-scope samples that arise
in multi-turn conversational interactions. We introduce MIntRec2.0, a
large-scale benchmark dataset for multimodal intent recognition in multi-party
conversations. It contains 1,245 dialogues with 15,040 samples, each annotated
within a new intent taxonomy of 30 fine-grained classes. Besides 9,304 in-scope
samples, it also includes 5,736 out-of-scope samples appearing in multi-turn
contexts, which naturally occur in real-world scenarios. Furthermore, we
provide comprehensive information on the speakers in each utterance, enriching
its utility for multi-party conversational research. We establish a general
framework supporting the organization of single-turn and multi-turn dialogue
data, modality feature extraction, multimodal fusion, as well as in-scope
classification and out-of-scope detection. Evaluation benchmarks are built
using classic multimodal fusion methods, ChatGPT, and human evaluators. While
existing methods incorporating nonverbal information yield improvements,
effectively leveraging context information and detecting out-of-scope samples
remains a substantial challenge. Notably, large language models exhibit a
significant performance gap compared to humans, highlighting the limitations of
machine learning methods in the cognitive intent understanding task. We believe
that MIntRec2.0 will serve as a valuable resource, providing a pioneering
foundation for research in human-machine conversational interactions, and
significantly facilitating related applications. The full dataset and codes are
available at this https URL

MIntRec2.0 是一个大规模的多模态意图识别基准数据集，包含 1,245 个对话，15,040 个样本，使用 30 个细粒度类别的新意图分类法进行注释，还包括 5,736 个出现在多轮对话上下文中的超范围样本。