In this article, we present our methodologies for SemEval-2021 Task-4: Reading Comprehension of Abstract Meaning. Given a fill-in-the-blank-type question and a corresponding context, the task is to predict the most suitable word from a list of 5 options. There are three sub-tasks within this task: Imperceptibility (subtask-I), Non-Specificity (subtask-II), and Intersection (subtask-III). We use encoders of transformers-based models pre-trained on the masked language modelling (MLM) task to build our Fill-in-the-blank (FitB) models. Moreover, to model imperceptibility, we define certain linguistic features, and to model non-specificity, we leverage information from hypernyms and hyponyms provided by a lexical database. Specifically, for non-specificity, we try out augmentation techniques, and other statistical techniques. We also propose variants, namely Chunk Voting and Max Context, to take care of input length restrictions for BERT, etc. Additionally, we perform a thorough ablation study, and use Integrated Gradients to explain our predictions on a few samples. Our best submissions achieve accuracies of 75.31% and 77.84%, on the test sets for subtask-I and subtask-II, respectively. For subtask-III, we achieve accuracies of 65.64% and 62.27%.

本文介绍了我们在SemEval-2021任务4中的方法，使用transformers-based模型和词汇数据库进行阅读理解，包括用于模拟不可感知性的语言特征和用于模拟非特定性的信息，并提出了改进的变体，同时论文也介绍了一些对预测解释的技术。在测试集中，任务一和任务二的最佳成果为75.31％和77.84％，而任务三的最佳成果为65.64％和62.27％。

LRG参加SemEval-2021任务4: 使用增强、语言特征和投票改进抽象词的阅读理解