Interpretable entity representations (IERs) are sparse embeddings that are
"human-readable" in that dimensions correspond to fine-grained entity types and
values are predicted probabilities that a given entity is of the corresponding
type. These methods perform well in zero-shot and low supervision settings.
Compared to standard dense neural embeddings, such interpretable
representations may permit analysis and debugging. However, while fine-tuning
sparse, interpretable representations improves accuracy on downstream tasks, it
destroys the semantics of the dimensions which were enforced in pre-training.
Can we maintain the interpretable semantics afforded by IERs while improving
predictive performance on downstream tasks? Toward this end, we propose
Intermediate enTity-based Sparse Interpretable Representation Learning
(ItsIRL). ItsIRL realizes improved performance over prior IERs on biomedical
tasks, while maintaining "interpretability" generally and their ability to
support model debugging specifically. The latter is enabled in part by the
ability to perform "counterfactual" fine-grained entity type manipulation,
which we explore in this work. Finally, we propose a method to construct entity
type based class prototypes for revealing global semantic properties of classes
learned by our model.

本文介绍的 Intermediate enTity-based Sparse Interpretable Representation Learning (ItsIRL) 方法在生物医学任务中表现优异，同时维护了 “可解释性” 和支持模型调试的能力，并探索了 “反事实” 精细实体类型操纵的能力以及构造基于实体类型的类原型以揭示模型学习的类别的全局语义特征。