Attention models have been intensively studied to improve NLP tasks such as
machine comprehension via both question-aware passage attention model and
self-matching attention model. Our research proposes phase conductor
(PhaseCond) for attention models in two meaningful ways. First, PhaseCond, an
architecture of multi-layered attention models, consists of multiple phases
each implementing a stack of attention layers producing passage representations
and a stack of inner or outer fusion layers regulating the information flow.
Second, we extend and improve the dot-product attention function for PhaseCond
by simultaneously encoding multiple question and passage embedding layers from
different perspectives. We demonstrate the effectiveness of our proposed model
PhaseCond on the SQuAD dataset, showing that our model significantly
outperforms both state-of-the-art single-layered and multiple-layered attention
models. We deepen our results with new findings via both detailed qualitative
analysis and visualized examples showing the dynamic changes through
multi-layered attention models.

本研究提出了一种名为 PhaseCond 的多层关键词模型，其中包含多个实现通道表示和信息流调节的关注层堆栈和内部或外部融合层堆栈，并通过同时对来自不同角度的多个问题和通道嵌入层进行编码来扩展和改进 PhaseCond 的点积关注函数，结果表明该模型在 SQuAD 数据集上明显优于单层和多层注意模型。