Large scale neural models show impressive performance across a wide array of
linguistic tasks. Despite this they remain, largely, black-boxes - inducing
vector-representations of their input that prove difficult to interpret. This
limits our ability to understand what they learn, and when the learn it, or
describe what kinds of representations generalise well out of distribution. To
address this we introduce a novel approach to interpretability that looks at
the mapping a model learns from sentences to representations as a kind of
language in its own right. In doing so we introduce a set of
information-theoretic measures that quantify how structured a model's
representations are with respect to its input, and when during training that
structure arises. Our measures are fast to compute, grounded in linguistic
theory, and can predict which models will generalise best based on their
representations. We use these measures to describe two distinct phases of
training a transformer: an initial phase of in-distribution learning which
reduces task loss, then a second stage where representations becoming robust to
noise. Generalisation performance begins to increase during this second phase,
drawing a link between generalisation and robustness to noise. Finally we look
at how model size affects the structure of the representational space, showing
that larger models ultimately compress their representations more than their
smaller counterparts.

通过一种新颖的可解释性方法，该论文使用信息论度量量化了神经模型对输入的表示的结构化程度，从而预测模型的泛化能力，并发现模型的代表性和噪声强度有关。此外，研究还研究了模型大小对表示空间结构的影响。

表示形式作为语言：一个信息论解释的框架

Representations as Language: An Information-Theoretic Framework for  Interpretability

This paper describes a web-based corpus of global language use with a focus
on how this corpus can be used for data-driven language mapping. First, the
corpus provides a representation of where national varieties of major languages
are used (e.g., English, Arabic, Russian) together with consistently collected
data for each variety. Second, the paper evaluates a language identification
model that supports more local languages with smaller sample sizes than
alternative off-the-shelf models. Improved language identification is essential
for moving beyond majority languages. Given the focus on language mapping, the
paper analyzes how well this digital language data represents actual
populations by (i) systematically comparing the corpus with demographic
ground-truth data and (ii) triangulating the corpus with an alternate
Twitter-based dataset. In total, the corpus contains 423 billion words
representing 148 languages (with over 1 million words from each language) and
158 countries (again with over 1 million words from each country), all
distilled from Common Crawl web data. The main contribution of this paper, in
addition to describing this publicly-available corpus, is to provide a
comprehensive analysis of the relationship between two sources of digital data
(the web and Twitter) as well as their connection to underlying populations.

本文描述了一个基于互联网语言使用的语料库，重点介绍如何利用这个语料库进行数据驱动的语言映射；通过提供有关全球语言使用的信息来支持本地化语种，并且通过对比人口统计学数据和 Twitter 数据来分析数字语言数据代表实际人口的程度。