The commercialization of large language models (LLMs) has led to the common
practice of high-level API-only access to proprietary models. In this work, we
show that even with a conservative assumption about the model architecture, it
is possible to learn a surprisingly large amount of non-public information
about an API-protected LLM from a relatively small number of API queries (e.g.,
costing under $1,000 for OpenAI's gpt-3.5-turbo). Our findings are centered on
one key observation: most modern LLMs suffer from a softmax bottleneck, which
restricts the model outputs to a linear subspace of the full output space. We
show that this lends itself to a model image or a model signature which unlocks
several capabilities with affordable cost: efficiently discovering the LLM's
hidden size, obtaining full-vocabulary outputs, detecting and disambiguating
different model updates, identifying the source LLM given a single full LLM
output, and even estimating the output layer parameters. Our empirical
investigations show the effectiveness of our methods, which allow us to
estimate the embedding size of OpenAI's gpt-3.5-turbo to be about 4,096.
Lastly, we discuss ways that LLM providers can guard against these attacks, as
well as how these capabilities can be viewed as a feature (rather than a bug)
by allowing for greater transparency and accountability.

大规模语言模型（LLMs）的商业化导致了对专有模型进行高级 API 访问的常见做法。然而，我们的研究发现，即使在对模型架构保守假设的情况下，通过相对较少的 API 查询（例如，OpenAI 的 gpt-3.5-turbo 下成本不到 1000 美元），仍然可以从 API 保护的 LLM 中获取大量非公开信息。此研究发现的核心观察结果是：大多数现代 LLM 都存在 softmax 瓶颈，这会将模型输出限制在完整输出空间的线性子空间内。我们展示了这一现象可用于建立模型图像或模型特征，从而具备多种功能，包括高效发现 LLM 的隐藏大小、获取完整词汇输出、检测和消除不同的模型更新、根据单个完整 LLM 输出确定源 LLM，甚至估计输出层参数。我们的实证研究证明了我们方法的有效性，根据我们的方法，我们估计 OpenAI 的 gpt-3.5-turbo 的嵌入大小约为 4,096。最后，我们讨论了 LLM 供应商可以采取的保护措施，以及这些功能如何被视为一种特性（而非 Bug），通过实现更高的透明度和责任可得到更多好处。

API 保护的 LLMs 泄露了专有信息

Logits of API-Protected LLMs Leak Proprietary Information

Coreference Resolution is an important NLP task and most state-of-the-art
methods rely on word embeddings for word representation. However, one issue
that has been largely overlooked in literature is that of comparing the
performance of different embeddings across and within families in this task.
Therefore, we frame our study in the context of Event and Entity Coreference
Resolution (EvCR & EnCR), and address two questions : 1) Is there a trade-off
between performance (predictive & run-time) and embedding size? 2) How do the
embeddings' performance compare within and across families? Our experiments
reveal several interesting findings. First, we observe diminishing returns in
performance with respect to embedding size. E.g. a model using solely a
character embedding achieves 86% of the performance of the largest model (Elmo,
GloVe, Character) while being 1.2% of its size. Second, the larger model using
multiple embeddings learns faster overall despite being slower per epoch.
However, it is still slower at test time. Finally, Elmo performs best on both
EvCR and EnCR, while GloVe and FastText perform best in EvCR and EnCR
respectively.

本研究在核指消解任务中探讨不同的词嵌入方法，研究表明，与嵌入维数相比，性能存在收益递减的情况，并且 Elmo 在 EvCR 和 EnCR 上表现最好，而 GloVe 和 FastText 分别在 EvCR 和 EnCR 中表现最佳。