Using the attention map based probing framework from (Clark et al., 2019), we observe that BERT's attention heads have modest but well above-chance ability to spot event arguments sans any training or domain finetuning, varying from a low of 17.77% for Place to a high of 51.61% for Artifact. Next, we find that linear combinations of these heads, estimated with approx 11% of available total supervision, can push performance well-higher for some roles - highest two being Victim (68.29% Accuracy) and Artifact (58.82% Accuracy). Furthermore, we investigate how well our methods do for cross-sentence event arguments. We propose a procedure to isolate "best heads" for cross-sentence argument detection separately of those for intra-sentence arguments. The heads thus estimated have superior cross-sentence performance compared to their jointly estimated equivalents. Lastly, we seek to isolate to what extent our numbers stem from lexical frequency based associations between gold arguments and roles. We propose NONCE, a scheme to create adversarial test examples by replacing gold arguments with randomly generated "nonce" words. We find that learnt linear combinations are robust to NONCE, though individual best heads can be much more sensitive.

使用注意力图探针框架，发现BERT模型对事件参数有良好的探测能力，并通过线性组合显著提高了某些角色的性能，尤其是在跨句子事件参数上表现更好。同时，通过提出的NONCE方法，可以验证单个最佳头部对词汇频率基础上的黄金参数和角色的关联是否具有鲁棒性。

BERTering RAMS：BERT模型已经掌握了多少事件参数知识？——以RAMS数据集为例