In recent times, large language models (LLMs) have made significant strides
in generating computer code, blurring the lines between code created by humans
and code produced by artificial intelligence (AI). As these technologies evolve
rapidly, it is crucial to explore how they influence code generation,
especially given the risk of misuse in areas like higher education. This paper
explores this issue by using advanced classification techniques to
differentiate between code written by humans and that generated by ChatGPT, a
type of LLM. We employ a new approach that combines powerful embedding features
(black-box) with supervised learning algorithms - including Deep Neural
Networks, Random Forests, and Extreme Gradient Boosting - to achieve this
differentiation with an impressive accuracy of 98%. For the successful
combinations, we also examine their model calibration, showing that some of the
models are extremely well calibrated. Additionally, we present white-box
features and an interpretable Bayes classifier to elucidate critical
differences between the code sources, enhancing the explainability and
transparency of our approach. Both approaches work well but provide at most
85-88% accuracy. We also show that untrained humans solve the same task not
better than random guessing. This study is crucial in understanding and
mitigating the potential risks associated with using AI in code generation,
particularly in the context of higher education, software development, and
competitive programming.

研究论文通过使用先进的分类技术来区分由人写的代码和由 ChatGPT 生成的代码，探讨了大型语言模型在代码生成中的影响，特别是在高等教育领域的潜在风险和对策。