As machine learning models become more general, we need to characterise them
in richer, more meaningful ways. We describe a method to infer the cognitive
profile of a system from diverse experimental data. To do so, we introduce
measurement layouts that model how task-instance features interact with system
capabilities to affect performance. These features must be triangulated in
complex ways to be able to infer capabilities from non-populational data -- a
challenge for traditional psychometric and inferential tools. Using the
Bayesian probabilistic programming library PyMC, we infer different cognitive
profiles for agents in two scenarios: 68 actual contestants in the AnimalAI
Olympics and 30 synthetic agents for O-PIAAGETS, an object permanence battery.
We showcase the potential for capability-oriented evaluation.

从多样的实验数据中推断系统的认知特征，我们介绍了一种方法来描述机器学习模型，引入测量布局来模拟任务实例特征与系统能力如何影响性能，使用贝叶斯概率编程库 PyMC 推断出不同认知特征的代理在动物人工智能奥林匹克的 68 名实际参赛选手和 O-PIAAGETS 的 30 个合成代理的能力，展示了基于能力的评估的潜力。