Article prediction is a task that has long defied accurate linguistic description. As such, this task is ideally suited to evaluate models on their ability to emulate native-speaker intuition. To this end, we compare the performance of native English speakers and pre-trained models on the task of article prediction set up as a three way choice (a/an, the, zero). Our experiments with BERT show that BERT outperforms humans on this task across all articles. In particular, BERT is far superior to humans at detecting the zero article, possibly because we insert them using rules that the deep neural model can easily pick up. More interestingly, we find that BERT tends to agree more with annotators than with the corpus when inter-annotator agreement is high but switches to agreeing more with the corpus as inter-annotator agreement drops. We contend that this alignment with annotators, despite being trained on the corpus, suggests that BERT is not memorising article use, but captures a high level generalisation of article use akin to human intuition.

本文通过比较以a/an，the和零三种形式设置的冠词预测任务中Bert模型和人类的表现，发现Bert模型对于检测零冠词的性能远远优于人类，并且在高一致性的情况下更容易与注释者达成一致，这表明BERT不是在记忆冠词使用情况，而是捕捉到了类似于人类的感性认知水平的高度概括性冠词使用规律。

抽象而非内存：BERT 与英语文章系统