In this work we study the presence of expert units in pre-trained Transformer
Models (TM), and how they impact a model's performance. We define expert units
to be neurons that are able to classify a concept with a given average
precision, where a concept is represented by a binary set of sentences
containing the concept (or not). Leveraging the OneSec dataset (Scarlini et
al., 2019), we compile a dataset of 1641 concepts that allows diverse expert
units in TM to be discovered. We show that expert units are important in
several ways: (1) The presence of expert units is correlated ($r^2=0.833$) with
the generalization power of TM, which allows ranking TM without requiring
fine-tuning on suites of downstream tasks. We further propose an empirical
method to decide how accurate such experts should be to evaluate
generalization. (2) The overlap of top experts between concepts provides a
sensible way to quantify concept co-learning, which can be used for
explainability of unknown concepts. (3) We show how to self-condition
off-the-shelf pre-trained language models to generate text with a given concept
by forcing the top experts to be active, without requiring re-training the
model or using additional parameters.

本研究旨在探讨预训练 Transformer 模型中的专家单元及其对模型性能的影响，并论证了这些单元在多个方面非常重要，如与模型泛化能力相关，可用于解释未知概念等，并通过强制激活关键专家单元的方法实现了对给定概念的文本生成，无需重新训练模型或使用附加参数。