Predictable behavior from scaling advanced AI systems is an extremely
desirable property. Although a well-established literature exists on how
pretraining performance scales, the literature on how particular downstream
capabilities scale is significantly muddier. In this work, we take a step back
and ask: why has predicting specific downstream capabilities with scale
remained elusive? While many factors are certainly responsible, we identify a
new factor that makes modeling scaling behavior on widely used multiple-choice
question-answering benchmarks challenging. Using five model families and twelve
well-established multiple-choice benchmarks, we show that downstream
performance is computed from negative log likelihoods via a sequence of
transformations that progressively degrade the statistical relationship between
performance and scale. We then reveal the mechanism causing this degradation:
downstream metrics require comparing the correct choice against a small number
of specific incorrect choices, meaning accurately predicting downstream
capabilities requires predicting not just how probability mass concentrates on
the correct choice with scale, but also how probability mass fluctuates on
specific incorrect choices with scale. We empirically study how probability
mass on the correct choice co-varies with probability mass on incorrect choices
with increasing compute, suggesting that scaling laws for incorrect choices
might be achievable. Our work also explains why pretraining scaling laws are
commonly regarded as more predictable than downstream capabilities and
contributes towards establishing scaling-predictable evaluations of frontier AI
models.

通过使用五个模型家族和十二个多项选择基准测试，我们展示了通过负对数似然性能计算的一系列转换逐渐降低了性能和规模之间的统计关系，同时揭示了导致这种降级的机制：下游度量要求将正确的选择与少量特定的错误选择进行比较，这意味着准确预测下游能力不仅要预测概率质量如何在正确选择上随规模集中，还要预测概率质量如何在特定错误选择上随规模波动。

为什么使用规模预测前沿的人工智能模型的下游能力如此困难？

Why Has Predicting Downstream Capabilities of Frontier AI Models with  Scale Remained Elusive?

Large Language Models (LLM) are prone to returning false information. It
constitutes one of major challenges in the AI field. In our work, we explore
paradigm introduced by Inference-Time-Intervention (ITI). In first stage, it
identifies attention heads, which contain the highest amount of desired type of
knowledge (e.g., truthful). Afterwards, during inference, LLM activations are
shifted for chosen subset of attention heads. We further improved the ITI
framework by introducing a nonlinear probing and multi-token intervention -
Non-Linear ITI (NL-ITI). NL-ITI is tested on diverse multiple-choice
benchmarks, including TruthfulQA, on which we report around 14% MC1 metric
improvement with respect to the baseline ITI results. NL-ITI achieves also
encouraging results on other testsets - on Business Ethics subdomain of MMLU,
around 18% MC1 improvement over baseline LLaMA2-7B. Additionally, NL-ITI
performs better while being less invasive in the behavior of LLM at the same
time (as measured by Kullback-Leibler divergence).

大型语言模型容易返回虚假信息，为了解决这个问题，该研究探索了一种介入推理时间的范式，引入了非线性探测和多标记介入方法，称之为 Non-Linear ITI。该方法在多种多项选择基准测试中取得了鼓舞人心的结果，并且在与其他基线和模型相比的实验中表现得更为非侵入性。