In this paper, we address the question answering challenge with the SQuAD 2.0
dataset. We design a model architecture which leverages BERT's capability of
context-aware word embeddings and BiDAF's context interactive exploration
mechanism. By integrating these two state-of-the-art architectures, our system
tries to extract the contextual word representation at word and character
levels, for better comprehension of both question and context and their
correlations. We also propose our original joint posterior probability
predictor module and its associated loss functions. Our best model so far
obtains F1 score of 75.842% and EM score of 72.24% on the test PCE leaderboad.

本文中，我们提出了一种模型架构，该模型利用了 BERT 的上下文感知词嵌入和 BiDAF 的上下文交互探索机制，试图在单词和字符级别上提取上下文词表示以更好地理解问题和上下文及其相关性，并提出了我们的原创联合后验概率预测模块及其相关损失函数。我们的最佳模型在测试 PCE 排行榜上获得了 75.842％的 F1 分数和 72.24％的 EM 分数。

基于上下文的联合概率模型在问答系统中的应用

Contextual Aware Joint Probability Model Towards Question Answering  System

To answer the question in machine comprehension (MC) task, the models need to
establish the interaction between the question and the context. To tackle the
problem that the single-pass model cannot reflect on and correct its answer, we
present Ruminating Reader. Ruminating Reader adds a second pass of attention
and a novel information fusion component to the Bi-Directional Attention Flow
model (BiDAF). We propose novel layer structures that construct an query-aware
context vector representation and fuse encoding representation with
intermediate representation on top of BiDAF model. We show that a multi-hop
attention mechanism can be applied to a bi-directional attention structure. In
experiments on SQuAD, we find that the Reader outperforms the BiDAF baseline by
a substantial margin, and matches or surpasses the performance of all other
published systems.

提出了一种基于 BiDAF 模型的 Ruminating Reader，通过增加多层注意力机制和信息融合组件来解决单次模型无法反思和纠正答案的问题。实验结果表明该模型在 SQuAD 数据集上显著优于基线模型，且超过了所有其他已发表的系统的性能。