A problem with many current Large Language Model (LLM) driven spoken
dialogues is the response time. Some efforts such as Groq address this issue by
lightning fast processing of the LLM, but we know from the cognitive psychology
literature that in human-to-human dialogue often responses occur prior to the
speaker completing their utterance. No amount of delay for LLM processing is
acceptable if we wish to maintain human dialogue latencies. In this paper, we
discuss methods for understanding an utterance in close to real time and
generating a response so that the system can comply with human-level
conversational turn delays. This means that the information content of the
final part of the speaker's utterance is lost to the LLM. Using the Google
NaturalQuestions (NQ) database, our results show GPT-4 can effectively fill in
missing context from a dropped word at the end of a question over 60% of the
time. We also provide some examples of utterances and the impacts of this
information loss on the quality of LLM response in the context of an avatar
that is currently under development. These results indicate that a simple
classifier could be used to determine whether a question is semantically
complete, or requires a filler phrase to allow a response to be generated
within human dialogue time constraints.

通过分析谷歌自然问答数据库，本文展示了 GPT-4 可在超过 60％的时间内从问题末尾的遗漏词语有效填补丢失的上下文，并探讨了这种信息损失对发展中的角色模型质量的影响，同时提出了使用简单分类器来确定是否需要填充短语以满足人类对话时间限制的问题是否语义完整。