Recent advancements in AI have democratized its deployment as a healthcare
assistant. While pretrained models from large-scale visual and audio datasets
have demonstrably generalized to this task, surprisingly, no studies have
explored pretrained speech models, which, as human-originated sounds,
intuitively would share closer resemblance to lung sounds. This paper explores
the efficacy of pretrained speech models for respiratory sound classification.
We find that there is a characterization gap between speech and lung sound
samples, and to bridge this gap, data augmentation is essential. However, the
most widely used augmentation technique for audio and speech, SpecAugment,
requires 2-dimensional spectrogram format and cannot be applied to models
pretrained on speech waveforms. To address this, we propose RepAugment, an
input-agnostic representation-level augmentation technique that outperforms
SpecAugment, but is also suitable for respiratory sound classification with
waveform pretrained models. Experimental results show that our approach
outperforms the SpecAugment, demonstrating a substantial improvement in the
accuracy of minority disease classes, reaching up to 7.14%.

探索预训练语音模型在呼吸音分类中的有效性，并提出了能够处理预训练语音波形的输入不可知表示级别增广技术。实验结果表明，该方法优于 SpecAugment，在少数疾病类别的准确性上实现了显著改善，最高达到 7.14%。

RepAugment：面向输入的表示层数据增强用于呼吸音分类

RepAugment: Input-Agnostic Representation-Level Augmentation for  Respiratory Sound Classification

In Spoken language understanding (SLU), a natural solution is concatenating
pre-trained speech models (e.g. HuBERT) and pretrained language models (PLM,
e.g. T5). Most previous works use pretrained language models with subword-based
tokenization. However, the granularity of input units affects the alignment of
speech model outputs and language model inputs, and PLM with character-based
tokenization is underexplored. In this work, we conduct extensive studies on
how PLMs with different tokenization strategies affect spoken language
understanding task including spoken question answering (SQA) and speech
translation (ST). We further extend the idea to create T5lephone(pronounced as
telephone), a variant of T5 that is pretrained using phonemicized text. We
initialize T5lephone with existing PLMs to pretrain it using relatively
lightweight computational resources. We reached state-of-the-art on NMSQA, and
the T5lephone model exceeds T5 with other types of units on end-to-end SQA and
ST.

本文研究使用不同分词策略的预训练语言模型对口语语言理解任务的影响，特别是在语音问答和语音翻译方面。通过使用音素文本进行预训练，创建了一个名为 T5lephone 的变种，并取得了 NMSQA 的最新成果，同时在端到端问答和翻译中超越了其他类型单元的 T5 模型。