In this paper, we assess the efficacy of ChatGPT (version Feb 2023), a
large-scale language model, in solving probability problems typically presented
in introductory computer engineering exams. Our study comprised a set of 23
probability exercises administered to students at Rey Juan Carlos University
(URJC) in Madrid. The responses produced by ChatGPT were evaluated by a group
of five statistics professors, who assessed them qualitatively and assigned
grades based on the same criteria used for students. Our results indicate that
ChatGPT surpasses the average student in terms of phrasing, organization, and
logical reasoning. The model's performance remained consistent for both the
Spanish and English versions of the exercises. However, ChatGPT encountered
difficulties in executing basic numerical operations. Our experiments
demonstrate that requesting ChatGPT to provide the solution in the form of an R
script proved to be an effective approach for overcoming these limitations. In
summary, our results indicate that ChatGPT surpasses the average student in
solving probability problems commonly presented in introductory computer
engineering exams. Nonetheless, the model exhibits limitations in reasoning
around certain probability concepts. The model's ability to deliver
high-quality explanations and illustrate solutions in any programming language,
coupled with its performance in solving probability exercises, suggests that
large language models have the potential to serve as learning assistants.

我们评估了 ChatGPT（2023 年 2 月版本），即一个大规模语言模型，在解决典型的介绍性计算机工程考试中出现的概率问题方面的效果。我们的研究包括了一套 23 个概率练习，这些练习被用来测试马德里市 Rey Juan Carlos 大学（URJC）的学生。我们对 ChatGPT 生成的回答进行了定性评估，并根据与学生相同的标准评分。我们的结果表明，ChatGPT 在措辞、组织和逻辑推理方面超过了平均学生。该模型在西班牙语和英语版本的练习中表现一致。然而，ChatGPT 在执行基本的数值运算方面遇到了困难。我们的实验表明，要求 ChatGPT 以 R 脚本形式提供解决方案是克服这些限制的有效方法。总之，我们的结果表明，ChatGPT 在解决计算机工程入门考试中常见的概率问题方面超过了平均学生。然而，该模型在某些概率概念的推理方面存在局限性。大型语言模型在提供高质量解释和以任何编程语言呈现解决方案方面的能力，以及其在解决概率练习中的表现，显示了它们作为学习助理的潜力。