The recent breakthrough successes in machine learning are mainly attributed
to scale: namely large-scale attention-based architectures and datasets of
unprecedented scale. This paper investigates the impact of training at scale
for chess. Unlike traditional chess engines that rely on complex heuristics,
explicit search, or a combination of both, we train a 270M parameter
transformer model with supervised learning on a dataset of 10 million chess
games. We annotate each board in the dataset with action-values provided by the
powerful Stockfish 16 engine, leading to roughly 15 billion data points. Our
largest model reaches a Lichess blitz Elo of 2895 against humans, and
successfully solves a series of challenging chess puzzles, without any
domain-specific tweaks or explicit search algorithms. We also show that our
model outperforms AlphaZero's policy and value networks (without MCTS) and
GPT-3.5-turbo-instruct. A systematic investigation of model and dataset size
shows that strong chess performance only arises at sufficient scale. To
validate our results, we perform an extensive series of ablations of design
choices and hyperparameters.

研究表明，机器学习中的重要突破主要归功于规模，特别是大规模的基于注意力的架构和空前规模的数据集。本文通过在一个包含一千万局国际象棋比赛的数据集上对一个含有 2.7 亿个参数的 transformer 模型进行监督学习，利用由强大的 Stockfish 16 引擎提供的动作价值为数据集中的每个棋盘进行注释，得到了约 150 亿个数据点，并达到了 2895 的 Lichess 快棋等级分，成功解决了一系列有难度的国际象棋难题，而无需特定领域的优化或显式搜索算法。研究还表明，我们的模型在性能上超过了 AlphaZero 的策略和值网络（没有 MCTS）以及 GPT-3.5-turbo-instruct。对模型和数据集规模进行的系统研究表明，只有在足够规模的情况下，才能展现强大的国际象棋性能。为了验证我们的结果，我们进行了一系列关于设计选择和超参数的消融实验。