Chat models are capable of answering a wide range of questions, however, the
accuracy of their responses is highly uncertain. In this research, we propose a
specialized PEFT-MedAware model where we utilize parameter-efficient
fine-tuning (PEFT) to enhance the Falcon-1b large language model on specialized
MedQuAD data consisting of 16,407 medical QA pairs, leveraging only 0.44% of
its trainable parameters to enhance computational efficiency. The paper adopts
data preprocessing and PEFT to optimize model performance, complemented by a
BitsAndBytesConfig for efficient transformer training. The resulting model was
capable of outperforming other LLMs in medical question-answering tasks in
specific domains with greater accuracy utilizing limited computational
resources making it suitable for deployment in resource-constrained
environments. We propose further improvements through expanded datasets, larger
models, and feedback mechanisms for sustained medical relevancy. Our work
highlights the efficiency gains and specialized capabilities of PEFT in medical
AI, outpacing standard models in precision without extensive resource demands.
The proposed model and data are released for research purposes only.

利用参数高效微调（PEFT）技术，本研究提出了一种专用的 PEFT-MedAware 模型，使用 16,407 个医学问答对的特定数据对 Falcon-1b 大型语言模型进行优化，仅利用其 0.44% 的可训练参数，提高计算效率。该模型通过数据预处理和 PEFT 来优化性能，并结合 BitsAndBytesConfig 进行高效的 Transformer 训练。结果表明，该模型在资源有限的环境下，在特定领域的医学问答任务中具有更高的准确性，能够胜过其他语言模型，展示了 PEFT 在医疗 AI 中的高效性和专业能力。建议通过扩展数据集、使用更大的模型和引入反馈机制来进一步改进，以保持医疗相关性。所提出的模型和数据仅供研究目的使用。