In this study, we use the existing Large Language Models ENnhanced to See
Framework (LENS Framework) to test the feasibility of multimodal task-oriented
dialogues. The LENS Framework has been proposed as a method to solve computer
vision tasks without additional training and with fixed parameters of
pre-trained models. We used the Multimodal Dialogs (MMD) dataset, a multimodal
task-oriented dialogue benchmark dataset from the fashion field, and for the
evaluation, we used the ChatGPT-based G-EVAL, which only accepts textual
modalities, with arrangements to handle multimodal data. Compared to
Transformer-based models in previous studies, our method demonstrated an
absolute lift of 10.8% in fluency, 8.8% in usefulness, and 5.2% in relevance
and coherence. The results show that using large-scale models with fixed
parameters rather than using models trained on a dataset from scratch improves
performance in multimodal task-oriented dialogues. At the same time, we show
that Large Language Models (LLMs) are effective for multimodal task-oriented
dialogues. This is expected to lead to efficient applications to existing
systems.

使用现有的 LENS Framework 来测试对话系统的多模态任务的可行性，与以往的基于 Transformer 的模型相比，我们的方法在流畅性、有用性以及相关性和连贯性上均有显著提升，同时证明了大规模预训练模型对于多模态任务导向对话系统的有效性。