Respiratory sound classification (RSC) is challenging due to varied acoustic
signatures, primarily influenced by patient demographics and recording
environments. To address this issue, we introduce a text-audio multimodal model
that utilizes metadata of respiratory sounds, which provides useful
complementary information for RSC. Specifically, we fine-tune a pretrained
text-audio multimodal model using free-text descriptions derived from the sound
samples' metadata which includes the gender and age of patients, type of
recording devices, and recording location on the patient's body. Our method
achieves state-of-the-art performance on the ICBHI dataset, surpassing the
previous best result by a notable margin of 1.17%. This result validates the
effectiveness of leveraging metadata and respiratory sound samples in enhancing
RSC performance. Additionally, we investigate the model performance in the case
where metadata is partially unavailable, which may occur in real-world clinical
setting.

通过利用呼吸音样本的元数据，我们引入了一个文本 - 音频多模态模型来改进呼吸音分类的性能，并且在治疗临床环境中部分元数据不可用的情况下也进行了研究验证。

BTS：基于元数据辅助的呼吸音分类的文本和声音模态的桥接

BTS: Bridging Text and Sound Modalities for Metadata-Aided Respiratory  Sound Classification

Recent advancements in AI have democratized its deployment as a healthcare
assistant. While pretrained models from large-scale visual and audio datasets
have demonstrably generalized to this task, surprisingly, no studies have
explored pretrained speech models, which, as human-originated sounds,
intuitively would share closer resemblance to lung sounds. This paper explores
the efficacy of pretrained speech models for respiratory sound classification.
We find that there is a characterization gap between speech and lung sound
samples, and to bridge this gap, data augmentation is essential. However, the
most widely used augmentation technique for audio and speech, SpecAugment,
requires 2-dimensional spectrogram format and cannot be applied to models
pretrained on speech waveforms. To address this, we propose RepAugment, an
input-agnostic representation-level augmentation technique that outperforms
SpecAugment, but is also suitable for respiratory sound classification with
waveform pretrained models. Experimental results show that our approach
outperforms the SpecAugment, demonstrating a substantial improvement in the
accuracy of minority disease classes, reaching up to 7.14%.

探索预训练语音模型在呼吸音分类中的有效性，并提出了能够处理预训练语音波形的输入不可知表示级别增广技术。实验结果表明，该方法优于 SpecAugment，在少数疾病类别的准确性上实现了显著改善，最高达到 7.14%。

RepAugment：面向输入的表示层数据增强用于呼吸音分类

RepAugment: Input-Agnostic Representation-Level Augmentation for  Respiratory Sound Classification

Deep neural networks have been applied to audio spectrograms for respiratory
sound classification. Existing models often treat the spectrogram as a
synthetic image while overlooking its physical characteristics. In this paper,
a Multi-View Spectrogram Transformer (MVST) is proposed to embed different
views of time-frequency characteristics into the vision transformer.
Specifically, the proposed MVST splits the mel-spectrogram into different sized
patches, representing the multi-view acoustic elements of a respiratory sound.
These patches and positional embeddings are then fed into transformer encoders
to extract the attentional information among patches through a self-attention
mechanism. Finally, a gated fusion scheme is designed to automatically weigh
the multi-view features to highlight the best one in a specific scenario.
Experimental results on the ICBHI dataset demonstrate that the proposed MVST
significantly outperforms state-of-the-art methods for classifying respiratory
sounds.

提出了一种 Multi-View Spectrogram Transformer (MVST) 模型，将深度神经网络应用于呼吸音频谱图的分类，通过将 mel-spectrogram 分为不同大小的 patches，使用 transformer encoders 提取 patches 之间的注意力信息，并设计了一种门控融合机制来加强多视图特征，在呼吸音分类任务上明显优于现有最先进方法。