Pioneering advancements in large language model-powered agents have
underscored the design pattern of multi-agent collaboration, demonstrating that
collective intelligence can surpass the capabilities of each individual.
Inspired by the neural scaling law, which posits that increasing neurons leads
to emergent abilities, this study investigates whether a similar principle
applies to increasing agents in multi-agent collaboration. Technically, we
propose multi-agent collaboration networks (MacNet), which utilize directed
acyclic graphs to organize agents and streamline their interactive reasoning
via topological ordering, with solutions derived from their dialogues.
Extensive experiments show that MacNet consistently outperforms baseline
models, enabling effective agent collaboration across various network
topologies and supporting cooperation among more than a thousand agents.
Notably, we observed a small-world collaboration phenomenon, where topologies
resembling small-world properties achieved superior performance. Additionally,
we identified a collaborative scaling law, indicating that normalized solution
quality follows a logistic growth pattern as scaling agents, with collaborative
emergence occurring much earlier than previously observed instances of neural
emergence. The code and data will be available at
this https URL

通过研究多智能体协作中增加智能体的性能以及神经扩展定律，本文提出了多智能体协作网络（MacNet）作为一种组织智能体并通过拓扑排序优化其交互推理的方法，实验证明 MacNet 在各种网络拓扑中表现优异，并且支持超过一千个智能体之间的协作；此外，还观察到拓扑结构类似小世界性质时协作效果更好，并提出协作扩展定律，表明协作性能随着智能体数量的增加而服从逻辑增长模式，且协作效果较神经扩展更早地出现。

基于大型语言模型的多智能体协作的扩展

Scaling Large-Language-Model-based Multi-Agent Collaboration

On a variety of tasks, the performance of neural networks predictably
improves with training time, dataset size and model size across many orders of
magnitude. This phenomenon is known as a neural scaling law. Of fundamental
importance is the compute-optimal scaling law, which reports the performance as
a function of units of compute when choosing model sizes optimally. We analyze
a random feature model trained with gradient descent as a solvable model of
network training and generalization. This reproduces many observations about
neural scaling laws. First, our model makes a prediction about why the scaling
of performance with training time and with model size have different power law
exponents. Consequently, the theory predicts an asymmetric compute-optimal
scaling rule where the number of training steps are increased faster than model
parameters, consistent with recent empirical observations. Second, it has been
observed that early in training, networks converge to their infinite-width
dynamics at a rate $1/\textit{width}$ but at late time exhibit a rate
$\textit{width}^{-c}$, where $c$ depends on the structure of the architecture
and task. We show that our model exhibits this behavior. Lastly, our theory
shows how the gap between training and test loss can gradually build up over
time due to repeated reuse of data.

神经网络的表现在训练时间、数据集大小和模型大小上预测性地提高，这一现象被称为神经缩放定律，而计算最优缩放定律则是将性能作为计算单元函数以选择模型大小来报告的；研究表明，神经网络在训练早期以 $1/	extit {width}$ 的速度收敛到无限宽度动力学，但在后期表现为 $	extit {width}^{-c}$ 的速度，其中 $c$ 取决于架构和任务的结构；此外，理论上显示了由于数据的重复重用，训练和测试损失之间的差距可以随时间逐渐增大。

神经缩放法则的动力学模型

A Dynamical Model of Neural Scaling Laws

Pretrained large language models (LLMs) are surprisingly effective at
performing zero-shot tasks, including time-series forecasting. However,
understanding the mechanisms behind such capabilities remains highly
challenging due to the complexity of the models. In this paper, we study LLMs'
ability to extrapolate the behavior of dynamical systems whose evolution is
governed by principles of physical interest. Our results show that LLaMA 2, a
language model trained primarily on texts, achieves accurate predictions of
dynamical system time series without fine-tuning or prompt engineering.
Moreover, the accuracy of the learned physical rules increases with the length
of the input context window, revealing an in-context version of neural scaling
law. Along the way, we present a flexible and efficient algorithm for
extracting probability density functions of multi-digit numbers directly from
LLMs.

预训练的大型语言模型（LLMs）在进行零 - shot 任务（包括时间序列预测）时表现出惊人的效果，本文研究了 LLMs 在从事受物理规律控制的动力系统情景下外推行为的能力，结果显示 LLaMA2 在无需微调或提示工程的情况下能准确预测动力系统时间序列，此外，学习到的物理规律的准确性随输入环境窗口长度的增加而增加，揭示了一种上下文版本的神经缩放定律，并提出了一种灵活高效的算法，可直接从 LLMs 中提取多位数的概率密度函数。