Large language models are capable of displaying a wide range of abilities
that are not directly connected with the task for which they are trained:
predicting the next words of human-written texts. In this article, I discuss
the nature of this indirect acquisition process and its relation to other known
indirect processes. I argue that an important side effect of such indirect
acquisition is the development of integrated abilities. I discuss the extent to
which the abilities developed by large language models are predictable.
Finally, I briefly discuss the relation between the cognitive skills acquired
by these systems and human cognition.

大型语言模型的能力不仅局限于预测人类书写文本的下一句，还包括间接获取能力、综合能力的发展、可预测性以及与人类认知的关系。

关于大型语言模型的意外能力

On the Unexpected Abilities of Large Language Models

What is a good vector representation of an object? We believe that it should
be generative in 3D, in the sense that it can produce new 3D objects; as well
as be predictable from 2D, in the sense that it can be perceived from 2D
images. We propose a novel architecture, called the TL-embedding network, to
learn an embedding space with these properties. The network consists of two
components: (a) an autoencoder that ensures the representation is generative;
and (b) a convolutional network that ensures the representation is predictable.
This enables tackling a number of tasks including voxel prediction from 2D
images and 3D model retrieval. Extensive experimental analysis demonstrates the
usefulness and versatility of this embedding.

本文介绍了一种名为 TL 嵌入网络的神经网络结构，它可生成新的 3D 物体并从 2D 图像中预测，旨在获得一个既具有生成性，又具有可预测性的嵌入空间，该嵌入空间可用于多个任务，包括从 2D 图像预测体素和 3D 模型检索。

学习物体的可预测和生成向量表示

Learning a Predictable and Generative Vector Representation for Objects

In this paper we study output coding for multi-label prediction. For a
multi-label output coding to be discriminative, it is important that codewords
for different label vectors are significantly different from each other. In the
meantime, unlike in traditional coding theory, codewords in output coding are
to be predicted from the input, so it is also critical to have a predictable
label encoding.
To find output codes that are both discriminative and predictable, we first
propose a max-margin formulation that naturally captures these two properties.
We then convert it to a metric learning formulation, but with an exponentially
large number of constraints as commonly encountered in structured prediction
problems. Without a label structure for tractable inference, we use
overgenerating (i.e., relaxation) techniques combined with the cutting plane
method for optimization.
In our empirical study, the proposed output coding scheme outperforms a
variety of existing multi-label prediction methods for image, text and music
classification.

本文研究了用于多标签预测的输出编码问题，并提出了一种同时具有差异性和可预测性的最大间隔度量学习方法，该方法在图像、文本和音乐分类方面的实证研究中优于其他多标签预测方法。