Sentence embeddings induced with various transformer architectures encode
much semantic and syntactic information in a distributed manner in a
one-dimensional array. We investigate whether specific grammatical information
can be accessed in these distributed representations. Using data from a task
developed to test rule-like generalizations, our experiments on detecting
subject-verb agreement yield several promising results. First, we show that
while the usual sentence representations encoded as one-dimensional arrays do
not easily support extraction of rule-like regularities, a two-dimensional
reshaping of these vectors allows various learning architectures to access such
information. Next, we show that various architectures can detect patterns in
these two-dimensional reshaped sentence embeddings and successfully learn a
model based on smaller amounts of simpler training data, which performs well on
more complex test data. This indicates that current sentence embeddings contain
information that is regularly distributed, and which can be captured when the
embeddings are reshaped into higher dimensional arrays. Our results cast light
on representations produced by language models and help move towards developing
few-shot learning approaches.

通过使用来自语言模型的更高维数组，我们发现句子嵌入可捕获规则类的语法信息，进而帮助发展少样本学习方法。

BERT 句子嵌入中的语法信息作为二维数组

Grammatical information in BERT sentence embeddings as two-dimensional  arrays

Named entity recognition and relation classification are key stages for
extracting information from unstructured text. Several natural language
processing applications utilize the two tasks, such as information retrieval,
knowledge graph construction and completion, question answering and other
domain-specific applications, such as biomedical data mining. We present a
survey of recent approaches in the two tasks with focus on few-shot learning
approaches. Our work compares the main approaches followed in the two
paradigms. Additionally, we report the latest metric scores in the two tasks
with a structured analysis that considers the results in the few-shot learning
scope.

最近的方法调查了命名实体识别和关系分类这两个任务，重点关注少样本学习方法，并比较了两个任务中的主要方法，报告了少样本学习范围内的最新指标得分。