With the recent emergence of powerful instruction-tuned large language models
(LLMs), various helpful conversational Artificial Intelligence (AI) systems
have been deployed across many applications. When prompted by users, these AI
systems successfully perform a wide range of tasks as part of a conversation.
To provide some sort of memory and context, such approaches typically condition
their output on the entire conversational history. Although this sensitivity to
the conversational history can often lead to improved performance on subsequent
tasks, we find that performance can in fact also be negatively impacted, if
there is a task-switch. To the best of our knowledge, our work makes the first
attempt to formalize the study of such vulnerabilities and interference of
tasks in conversational LLMs caused by task-switches in the conversational
history. Our experiments across 5 datasets with 15 task switches using popular
LLMs reveal that many of the task-switches can lead to significant performance
degradation.

通过实验发现，在使用大型语言模型进行对话时，任务切换会导致性能下降。