The opacity of deep neural networks remains a challenge in deploying
solutions where explanation is as important as precision. We present ConceptX,
a human-in-the-loop framework for interpreting and annotating latent
representational space in pre-trained Language Models (pLMs). We use an
unsupervised method to discover concepts learned in these models and enable a
graphical interface for humans to generate explanations for the concepts. To
facilitate the process, we provide auto-annotations of the concepts (based on
traditional linguistic ontologies). Such annotations enable development of a
linguistic resource that directly represents latent concepts learned within
deep NLP models. These include not just traditional linguistic concepts, but
also task-specific or sensitive concepts (words grouped based on gender or
religious connotation) that helps the annotators to mark bias in the model. The
framework consists of two parts (i) concept discovery and (ii) annotation
platform.

该研究提出了 ConceptX 人在循环框架，用于解释深度语言模型中预训练隐藏表达空间，并提供自动生成的基于语言本体论的概念注释，以表示这些模型中学习到的隐含概念，从而帮助注释人员标记模型中的偏见。