Using a novel professional certification survey, the study focuses on
assessing the vocational skills of two highly cited AI models, GPT-3 and
Turbo-GPT3.5. The approach emphasizes the importance of practical readiness
over academic performance by examining the models' performances on a benchmark
dataset consisting of 1149 professional certifications. This study also
includes a comparison with human test scores, providing perspective on the
potential of AI models to match or even surpass human performance in
professional certifications. GPT-3, even without any fine-tuning or exam
preparation, managed to achieve a passing score (over 70% correct) on 39% of
the professional certifications. It showcased proficiency in computer-related
fields, including cloud and virtualization, business analytics, cybersecurity,
network setup and repair, and data analytics. Turbo-GPT3.5, on the other hand,
scored a perfect 100% on the highly regarded Offensive Security Certified
Professional (OSCP) exam. This model also demonstrated competency in diverse
professional fields, such as nursing, licensed counseling, pharmacy, and
aviation. Turbo-GPT3.5 exhibited strong performance on customer service tasks,
indicating potential use cases in enhancing chatbots for call centers and
routine advice services. Both models also scored well on sensory and
experience-based tests outside a machine's traditional roles, including wine
sommelier, beer tasting, emotional quotient, and body language reading. The
study found that OpenAI's model improvement from Babbage to Turbo led to a 60%
better performance on the grading scale within a few years. This progress
indicates that addressing the current model's limitations could yield an AI
capable of passing even the most rigorous professional certifications.

通过一项新型的专业认证调查，本研究重点评估了两个引用率较高的 AI 模型 GPT-3 和 Turbo-GPT3.5 的职业技能。该研究强调实践准备的重要性，通过考察模型在由 1149 个专业认证组成的基准数据集上的表现，与人类测试成绩进行比较，从而提供了 AI 模型在专业认证方面与甚至超过人类表现的潜力的视角。GPT-3 甚至在没有任何微调或考试准备的情况下，在 39% 的专业认证中获得了及格分数（超过 70% 的正确率），并在云计算与虚拟化、业务分析、网络设置和修复以及数据分析等计算机相关领域展示了熟练程度。另一方面，Turbo-GPT3.5 在备受推崇的 Offensive Security Certified Professional (OSCP) 考试上获得了满分的 100%。该模型还展示了在护理、许可咨询、药剂和航空等多个专业领域的能力。Turbo-GPT3.5 在客户服务任务上表现出色，表明在增强呼叫中心的聊天机器人和日常咨询服务方面具有潜在应用场景。两个模型在机器传统角色之外的感官和基于经验的测试中也表现良好，包括品酒师、啤酒品尝、情商以及身体语言解读。研究发现，OpenAI 从 Babbage 到 Turbo 的模型改进使得评分标准的性能提升了 60%。这一进展表明，解决现有模型的局限性可能会产生能够通过最严格的专业认证的 AI。

通过专业测试评估人工智能职业技能

Evaluating AI Vocational Skills Through Professional Testing

The research creates a professional certification survey to test large
language models and evaluate their employable skills. It compares the
performance of two AI models, GPT-3 and Turbo-GPT3.5, on a benchmark dataset of
1149 professional certifications, emphasizing vocational readiness rather than
academic performance. GPT-3 achieved a passing score (>70% correct) in 39% of
the professional certifications without fine-tuning or exam preparation. The
models demonstrated qualifications in various computer-related fields, such as
cloud and virtualization, business analytics, cybersecurity, network setup and
repair, and data analytics. Turbo-GPT3.5 scored 100% on the valuable Offensive
Security Certified Professional (OSCP) exam. The models also displayed
competence in other professional domains, including nursing, licensed
counseling, pharmacy, and teaching. Turbo-GPT3.5 passed the Financial Industry
Regulatory Authority (FINRA) Series 6 exam with a 70% grade without
preparation. Interestingly, Turbo-GPT3.5 performed well on customer service
tasks, suggesting potential applications in human augmentation for chatbots in
call centers and routine advice services. The models also score well on sensory
and experience-based tests such as wine sommelier, beer taster, emotional
quotient, and body language reader. The OpenAI model improvement from Babbage
to Turbo resulted in a median 60% better-graded performance in less than a few
years. This progress suggests that focusing on the latest model's shortcomings
could lead to a highly performant AI capable of mastering the most demanding
professional certifications. We open-source the benchmark to expand the range
of testable professional skills as the models improve or gain emergent
capabilities.

该研究设计了一个专业认证测试来评估大型语言模型的雇佣能力并比较了两个 AI 模型， GPT-3 和 Turbo-GPT3.5，该模型在 1149 个专业证书的基准数据集上表现出各种计算机相关领域的资格，模型的进步表明关注最新模型的缺点可能会引领人工智能技术超脱对最苛刻的职业认证的挑战。