What can contemporary machine learning (ML) models do? Given the
proliferation of ML models in society, answering this question matters to a
variety of stakeholders, both public and private. The evaluation of models'
capabilities is rapidly emerging as a key subfield of modern ML, buoyed by
regulatory attention and government grants. Despite this, the notion of an ML
model possessing a capability has not been interrogated: what are we saying
when we say that a model is able to do something? And what sorts of evidence
bear upon this question? In this paper, we aim to answer these questions, using
the capabilities of large language models (LLMs) as a running example. Drawing
on the large philosophical literature on abilities, we develop an account of ML
models' capabilities which can be usefully applied to the nascent science of
model evaluation. Our core proposal is a conditional analysis of model
abilities (CAMA): crudely, a machine learning model has a capability to X just
when it would reliably succeed at doing X if it 'tried'. The main contribution
of the paper is making this proposal precise in the context of ML, resulting in
an operationalisation of CAMA applicable to LLMs. We then put CAMA to work,
showing that it can help make sense of various features of ML model evaluation
practice, as well as suggest procedures for performing fair inter-model
comparisons.

在本文中，我们利用大语言模型（LLMs）的能力作为运行示例，借鉴能力的大量哲学文献，发展了一个能有效应用于模型评估的机器学习模型能力的条件分析（CAMA）的核心提案，在机器学习的背景下对 CAMA 进行了精确定义，从而实现了适用于 LLMs 的 CAMA 操作。然后，我们运用 CAMA，展示它能够有助于理解机器学习模型评估实践的各种特征，并提出了进行公正的模型间比较的程序。