Language model prompt optimization research has shown that semantically and
grammatically well-formed manually crafted prompts are routinely outperformed
by automatically generated token sequences with no apparent meaning or
syntactic structure, including sequences of vectors from a model's embedding
space. We use machine-generated prompts to probe how models respond to input
that is not composed of natural language expressions. We study the behavior of
models of different sizes in multiple semantic tasks in response to both
continuous and discrete machine-generated prompts, and compare it to the
behavior in response to human-generated natural-language prompts. Even when
producing a similar output, machine-generated and human prompts trigger
different response patterns through the network processing pathways, including
different perplexities, different attention and output entropy distributions,
and different unit activation profiles. We provide preliminary insight into the
nature of the units activated by different prompt types, suggesting that only
natural language prompts recruit a genuinely linguistic circuit.

语言模型提示优化研究表明，通过无明显意义或语法结构的自动生成的令牌序列，包括模型嵌入空间中的向量序列，通常胜过语义和语法良好的手工制作的提示。我们使用机器生成的提示来探究模型对非自然语言表达组成的输入的响应，并在多个语义任务中研究不同尺寸模型的行为，以及它们对连续和离散机器生成的提示和人工生成的自然语言提示的响应行为进行比较。即使产生相似的输出，机器生成的和人工提示通过网络处理途径触发不同的响应模式，包括不同的困惑度、不同的注意力和输出熵分布，以及不同的单元激活特征。我们提供了对不同提示类型激活的单元性质的初步洞察，表明只有自然语言提示才会引起真正的语言电路的激活。