Embedding text sequences is a widespread requirement in modern language
understanding. Existing approaches focus largely on constant-size
representations. This is problematic, as the amount of information contained in
text often varies with the length of the input. We propose a solution called
Nugget, which encodes language into a representation based on a dynamically
selected subset of input tokens. These nuggets are learned through tasks like
autoencoding and machine translation, and intuitively segment language into
meaningful units. We demonstrate Nugget outperforms related approaches in tasks
involving semantic comparison. Finally, we illustrate these compact units allow
for expanding the contextual window of a language model (LM), suggesting new
future LMs that can condition on significantly larger amounts of content.

Nugget 是一种将语言编码为基于动态选择的子集输入标记的表示方法，通过自动编码和机器翻译等任务学习这些紧凑单元来直观地将语言分成有意义的单元，并在涉及语义比较的任务中明显优于相关方法，同时该方法还允许扩大语言模型的上下文窗口，为未来的语言模型打开了新的前景。

Nugget：文本的神经集成嵌入

Nugget: Neural Agglomerative Embeddings of Text

Most surrogate models for computer experiments are interpolators, and the
most common interpolator is a Gaussian process (GP) that deliberately omits a
small-scale (measurement) error term called the nugget. The explanation is that
computer experiments are, by definition, "deterministic", and so there is no
measurement error. We think this is too narrow a focus for a computer
experiment and a statistically inefficient way to model them. We show that
estimating a (non-zero) nugget can lead to surrogate models with better
statistical properties, such as predictive accuracy and coverage, in a variety
of common situations.

本文探讨了计算机实验的代理模型中高斯过程（GP）的插值器存在的弊端，提出在代理模型中引入测量误差项（nugget）能够提高预测准确性和覆盖率的统计特性。