This paper introduces the schemes of Team LingJing's experiments in
NLPCC-2022-Shared-Task-4 Multi-modal Dialogue Understanding and Generation
(MDUG). The MDUG task can be divided into two phases: multi-modal context
understanding and response generation. To fully leverage the visual information
for both scene understanding and dialogue generation, we propose the
scene-aware prompt for the MDUG task. Specifically, we utilize the
multi-tasking strategy for jointly modelling the scene- and session-
multi-modal understanding. The visual captions are adopted to aware the scene
information, while the fixed-type templated prompt based on the scene- and
session-aware labels are used to further improve the dialogue generation
performance. Extensive experimental results show that the proposed method has
achieved state-of-the-art (SOTA) performance compared with other competitive
methods, where we rank the 1-st in all three subtasks in this MDUG competition.

本文介绍了 LingJing 团队在 NLPCC-2022-Shared-Task-4 多模态对话理解和生成中实验的方案，通过提出情景感知提示和多任务策略来联合建模情景和对话生成的多模态理解，在场景了解和对话生成方面实现了最先进的性能。