Reinforcement learning has been applied to train the dialog systems in many
works. Previous approaches divide the dialog system into multiple modules
including DST (dialog state tracking) and DP (dialog policy), and train these
modules simultaneously. However, different modules influence each other during
training. The errors from DST might misguide the dialog policy, and the system
action brings extra difficulties for the DST module. To alleviate this problem,
we propose Asynchronous Updating Reinforcement Learning framework (AURL) that
updates the DST module and the DP module asynchronously under a cooperative
setting. Furthermore, curriculum learning is implemented to address the problem
of unbalanced data distribution during reinforcement learning sampling, and
multiple user models are introduced to increase the dialog diversity. Results
on the public SSD-PHONE dataset show that our method achieves a compelling
result with a 31.37% improvement on the dialog success rate. The code is
publicly available via this https URL

提出了异步更新强化学习框架（AURL），通过协作设置异步更新 DST 模块和 DP 模块，并实现课程学习以解决强化学习采样过程中不平衡数据分布的问题，并引入多个用户模型增加对话的多样性，实验表明，在公共数据集 SSD-PHONE 上，该方法使对话成功率提高了 31.37%。

面向任务导向对话系统的异步更新强化学习框架

An Asynchronous Updating Reinforcement Learning Framework for  Task-oriented Dialog System

Recent advances in the design of neural network architectures, in particular
those specialized in modeling sequences, have provided significant improvements
in speech separation performance. In this work, we propose to use a
bio-inspired architecture called Fully Recurrent Convolutional Neural Network
(FRCNN) to solve the separation task. This model contains bottom-up, top-down
and lateral connections to fuse information processed at various time-scales
represented by \textit{stages}. In contrast to the traditional approach
updating stages in parallel, we propose to first update the stages one by one
in the bottom-up direction, then fuse information from adjacent stages
simultaneously and finally fuse information from all stages to the bottom stage
together. Experiments showed that this asynchronous updating scheme achieved
significantly better results with much fewer parameters than the traditional
synchronous updating scheme. In addition, the proposed model achieved good
balance between speech separation accuracy and computational efficiency as
compared to other state-of-the-art models on three benchmark datasets.

本论文提出采用一种名为全循环卷积神经网络（Fully Recurrent Convolutional Neural Network, FRCNN）的生物启发式体系结构来解决语音分离任务，并通过异步更新机制，在三个基准数据集上显著提高了语音分离精度和计算效率。