We present a generative dialogue system capable of operating in a full-duplex
manner, allowing for seamless interaction. It is based on a large language
model (LLM) carefully aligned to be aware of a perception module, a motor
function module, and the concept of a simple finite state machine (called
neural FSM) with two states. The perception and motor function modules operate
simultaneously, allowing the system to simultaneously speak and listen to the
user. The LLM generates textual tokens for inquiry responses and makes
autonomous decisions to start responding to, wait for, or interrupt the user by
emitting control tokens to the neural FSM. All these tasks of the LLM are
carried out as next token prediction on a serialized view of the dialogue in
real-time. In automatic quality evaluations simulating real-life interaction,
the proposed system reduces the average conversation response latency by more
than 3 folds compared with LLM-based half-duplex dialogue systems while
responding within less than 500 milliseconds in more than 50% of evaluated
interactions. Running a LLM with only 8 billion parameters, our system exhibits
a 8% higher interruption precision rate than the best available commercial LLM
for voice-based dialogue.

我们提出了一个基于大型语言模型的生成对话系统，能够以全双工方式运行，并同时处理对话中的听与说任务。