This study presents a novel methodology utilizing a pre-trained speech
recognition model for processing respiratory sound data. By incorporating
medical record information, we introduce an innovative multi-modal
deep-learning architecture, named Rene, which addresses the challenges of poor
interpretability and underperformance in real-time clinical diagnostic response
observed in previous respiratory disease-focused models. The proposed Rene
architecture demonstrated significant improvements of 10.24%, 16.15%, 15.29%,
and 18.90% respectively, compared to the baseline across four tasks related to
respiratory event detection and audio record classification on the SPRSound
database. In patient disease prediction tests on the ICBHI database, the
architecture exhibited improvements of 23% in the mean of average score and
harmonic score compared to the baseline. Furthermore, we developed a real-time
respiratory sound discrimination system based on the Rene architecture,
featuring a dual-thread design and compressed model parameters for simultaneous
microphone recording and real-time dynamic decoding. Employing state-of-the-art
Edge AI technology, this system enables rapid and accurate responses for
respiratory sound auscultation, facilitating deployment on wearable clinical
detection devices to capture incremental data, which can be synergistically
evolved with large-scale models deployed on cloud servers for downstream tasks.

利用预训练的语音识别模型处理呼吸音数据，通过引入医疗记录信息，提出了一种名为 Rene 的创新多模态深度学习架构，解决了以往呼吸疾病模型在实时临床诊断响应中存在的解释性和性能不足的挑战。在与基线相比的四项与呼吸事件检测和音频记录分类相关任务中，所提出的 Rene 架构分别取得了 10.24％，16.15％，15.29％和 18.90％的显著改进。在 ICBHI 数据库上进行的患者疾病预测测试中，该架构相对于基线在平均分数和谐波分数上分别提高了 23％。此外，我们基于 Rene 架构开发了实时呼吸音辨别系统，采用双线程设计和压缩模型参数用于同时麦克风录制和实时动态解码，利用最先进的边缘人工智能技术，该系统能够快速准确地响应呼吸音听诊，实现在可穿戴临床检测设备上的部署，以捕获增量数据，并能与在云服务器上部署的大规模模型协同发展用于下游任务。