Bidirectional Encoder Representations from Transformers (BERT) reach state-of-the-art results in a variety of Natural Language Processing tasks. However, understanding of their internal functioning is still insufficient and unsatisfactory. In order to better understand BERT and other Transformer-based models, we present a layer-wise analysis of BERT's hidden states. Unlike previous research, which mainly focuses on explaining Transformer models by their attention weights, we argue that hidden states contain equally valuable information. Specifically, our analysis focuses on models fine-tuned on the task of Question Answering (QA) as an example of a complex downstream task. We inspect how QA models transform token vectors in order to find the correct answer. To this end, we apply a set of general and QA-specific probing tasks that reveal the information stored in each representation layer. Our qualitative analysis of hidden state visualizations provides additional insights into BERT's reasoning process. Our results show that the transformations within BERT go through phases that are related to traditional pipeline tasks. The system can therefore implicitly incorporate task-specific information into its token representations. Furthermore, our analysis reveals that fine-tuning has little impact on the models' semantic abilities and that prediction errors can be recognized in the vector representations of even early layers.

我们对BERT的隐藏状态进行逐层分析，揭示了这些状态所包含的有价值的信息，包括QA任务fine-tuned的模型如何转换令牌向量以找到正确的答案。通过应用一组用于揭示每个表示层中存储信息的一般和QA特定探测任务，我们的分析显示BERT的转换经历了与传统pipeline任务相关的阶段，且细微调整对模型的语意能力影响不大，即使在早期层也可以识别出预测错误。

BERT是如何回答问题的：对Transformer表示进行逐层分析