Objective: Clinical deep phenotyping plays a critical role in both the
diagnosis of patients with rare disorders as well as in building care
coordination plans. The process relies on modelling and curating patient
profiles using ontology concepts, usually from the Human Phenotype Ontology.
Machine learning methods have been widely adopted to support this phenotype
concept recognition task. With the significant shift in the use of large
language models (LLMs) for most NLP tasks, herewithin, we examine the
performance of the latest Generative Pre-trained Transformer (GPT) models
underpinning ChatGPT in clinical deep phenotyping. Materials and Methods: The
experimental setup of the study included seven prompts of various levels of
specificity, two GPT models (gpt-3.5 and gpt-4.0) and an established gold
standard for phenotype recognition. Results: Our results show that, currently,
these models have not yet achieved state of the art performance. The best run,
using few-shots learning, achieved 0.41 F1 score, compared to a 0.62 F1 score
achieved by the current best in class tool. Conclusion: The non-deterministic
nature of the outcomes and the lack of concordance between different runs using
the same prompt and input makes the use of these LLMs in clinical settings
problematic.

临床深度表型化和本体概念建模在罕见疾病患者的诊断和护理协调计划中起着重要作用。本研究使用大型语言模型评估了最新的生成预训练变压器模型在临床深度表型化的性能，结果表明这些模型在临床环境中的使用存在问题。