Understanding geometric properties of natural language processing models'
latent spaces allows the manipulation of these properties for improved
performance on downstream tasks. One such property is the amount of data spread
in a model's latent space, or how fully the available latent space is being
used. In this work, we define data spread and demonstrate that the commonly
used measures of data spread, Average Cosine Similarity and a partition
function min/max ratio I(V), do not provide reliable metrics to compare the use
of latent space across models. We propose and examine eight alternative
measures of data spread, all but one of which improve over these current
metrics when applied to seven synthetic data distributions. Of our proposed
measures, we recommend one principal component-based measure and one
entropy-based measure that provide reliable, relative measures of spread and
can be used to compare models of different sizes and dimensionalities.

本文提出并验证八种数据分布度量方法，相对于现有方法，其中大部分具有改进效果，建议使用一种基于主要成分的度量方法和一种基于熵的度量方法来评估模型的数据分布情况。