Hoffmann et al. (2022) propose three methods for estimating a compute-optimal
scaling law. We attempt to replicate their third estimation procedure, which
involves fitting a parametric loss function to a reconstruction of data from
their plots. We find that the reported estimates are inconsistent with their
first two estimation methods, fail at fitting the extracted data, and report
implausibly narrow confidence intervals--intervals this narrow would require
over 600,000 experiments, while they likely only ran fewer than 500. In
contrast, our rederivation of the scaling law using the third approach yields
results that are compatible with the findings from the first two estimation
procedures described by Hoffmann et al.

Hoffmann et al.（2022）提出了三种估计计算最优缩放定律的方法，我们试图复制他们的第三个估计过程，其中涉及将参数损失函数拟合到从图表中重建的数据。我们发现所报道的估计与他们的前两种估计方法不一致，无法对提取的数据进行拟合，并报告了不可信的窄置信区间 - 这种窄度的区间需要超过 60 万个实验，而他们可能只进行了不到 500 个。相比之下，我们使用第三种方法重新推导缩放定律的结果与 Hoffmann 等人描述的前两种估计过程的发现是一致的。

安哥拉兔规模化实验

Chinchilla Scaling: A replication attempt

This paper considers the problem of measure estimation under the barycentric
coding model (BCM), in which an unknown measure is assumed to belong to the set
of Wasserstein-2 barycenters of a finite set of known measures. Estimating a
measure under this model is equivalent to estimating the unknown barycentric
coordinates. We provide novel geometrical, statistical, and computational
insights for measure estimation under the BCM, consisting of three main
results. Our first main result leverages the Riemannian geometry of
Wasserstein-2 space to provide a procedure for recovering the barycentric
coordinates as the solution to a quadratic optimization problem assuming access
to the true reference measures. The essential geometric insight is that the
parameters of this quadratic problem are determined by inner products between
the optimal displacement maps from the given measure to the reference measures
defining the BCM. Our second main result then establishes an algorithm for
solving for the coordinates in the BCM when all the measures are observed
empirically via i.i.d. samples. We prove precise rates of convergence for this
algorithm -- determined by the smoothness of the underlying measures and their
dimensionality -- thereby guaranteeing its statistical consistency. Finally, we
demonstrate the utility of the BCM and associated estimation procedures in
three application areas: (i) covariance estimation for Gaussian measures; (ii)
image processing; and (iii) natural language processing.

本文考虑了基于质心编码模型（BCM）的测量估计问题，其中假设未知的度量归属于已知度量的有限组的 Wasserstein-2 重心集合。我们提供了测量 BCM 的新的几何、统计和计算方面的见解，并且提供了针对 BCM 的估计程序在三个应用领域中的实用性。