Deep neural models, in particular Transformer-based pre-trained language
models, require a significant amount of data to train. This need for data tends
to lead to problems when dealing with idiomatic multiword expressions (MWEs),
which are inherently less frequent in natural text. As such, this work explores
sample efficient methods of idiomaticity detection. In particular we study the
impact of Pattern Exploit Training (PET), a few-shot method of classification,
and BERTRAM, an efficient method of creating contextual embeddings, on the task
of idiomaticity detection. In addition, to further explore generalisability, we
focus on the identification of MWEs not present in the training data. Our
experiments show that while these methods improve performance on English, they
are much less effective on Portuguese and Galician, leading to an overall
performance about on par with vanilla mBERT. Regardless, we believe sample
efficient methods for both identifying and representing potentially idiomatic
MWEs are very encouraging and hold significant potential for future
exploration.

本篇研究旨在探究识别成语化多词表达的高效方法，通过 Pattern Exploit Training 和 BERTRAM 两种少样本分类方法，以及探索不在训练数据中的多词表达的识别，提高英语的表现，但对葡萄牙语和加利西亚语的效果较差。

习惯用语检测的高效样本方法

Sample Efficient Approaches for Idiomaticity Detection

Pretraining deep language models has led to large performance gains in NLP.
Despite this success, Schick and Sch\"utze (2020) recently showed that these
models struggle to understand rare words. For static word embeddings, this
problem has been addressed by separately learning representations for rare
words. In this work, we transfer this idea to pretrained language models: We
introduce BERTRAM, a powerful architecture based on BERT that is capable of
inferring high-quality embeddings for rare words that are suitable as input
representations for deep language models. This is achieved by enabling the
surface form and contexts of a word to interact with each other in a deep
architecture. Integrating BERTRAM into BERT leads to large performance
increases due to improved representations of rare and medium frequency words on
both a rare word probing task and three downstream tasks.

本文提出了一种名为 BERTRAM 的基于 BERT 的结构，用于为罕见词建立高质量的嵌入表示，从而提高深层语言模型在罕见词和中频词上的表现。实验表明，在罕见词探查任务和三种下游任务上，与 BERT 相结合可以显著提高罕见词和中频词的表示。