We propose a Perceiver-based sequence classifier to detect abnormalities in
speech reflective of several neurological disorders. We combine this classifier
with a Universal Speech Model (USM) that is trained (unsupervised) on 12
million hours of diverse audio recordings. Our model compresses long sequences
into a small set of class-specific latent representations and a factorized
projection is used to predict different attributes of the disordered input
speech. The benefit of our approach is that it allows us to model different
regions of the input for different classes and is at the same time data
efficient. We evaluated the proposed model extensively on a curated corpus from
the Mayo Clinic. Our model outperforms standard transformer (80.9%) and
perceiver (81.8%) models and achieves an average accuracy of 83.1%. With
limited task-specific data, we find that pretraining is important and
surprisingly pretraining with the unrelated automatic speech recognition (ASR)
task is also beneficial. Encodings from the middle layers provide a mix of both
acoustic and phonetic information and achieve best prediction results compared
to just using the final layer encodings (83.1% vs. 79.6%). The results are
promising and with further refinements may help clinicians detect speech
abnormalities without needing access to highly specialized speech-language
pathologists.

我们提出了一种基于感知器的序列分类器，用于检测反映多种神经系统疾病的语音异常。我们将这个分类器与一个训练有素（无监督训练）的通用语音模型（USM）结合起来，该模型在 1200 万小时的多样化音频记录上进行训练。我们的模型将长序列压缩为一组小的与类别相关的潜在表示，并使用分解投影来预测有序输入语音的不同属性。我们的方法的好处是，它允许我们对输入的不同区域进行不同类别的建模，同时具有数据效率。我们在 Mayo Clinic 的手稿语料库上对提出的模型进行了广泛评估。我们的模型的性能优于标准的 Transformer 模型（80.9%）和感知器模型（81.8%），平均准确率达到 83.1%。通过有限的任务特定数据，我们发现预训练是重要的，并且出乎意料的是，预训练与不相关的自动语音识别（ASR）任务也是有益的。中间层的编码提供了声学和语音信息的混合，并且相对于仅使用最终层的编码（83.1% 对比 79.6%），获得了最佳预测结果。这些结果有很大的潜力，并且在进一步的改进后或许可以帮助医生检测语音异常而无需接触高度专业的言语病理学家。

利用基于感知器的序列分类器和通用语音模型检测语音异常

Detecting Speech Abnormalities with a Perceiver-based Sequence  Classifier that Leverages a Universal Speech Model

Recently, large-scale Contrastive Language-Image Pre-training (CLIP) has
attracted unprecedented attention for its impressive zero-shot recognition
ability and excellent transferability to downstream tasks. However, CLIP is
quite data-hungry and requires 400M image-text pairs for pre-training, thereby
restricting its adoption. This work proposes a novel training paradigm, Data
efficient CLIP (DeCLIP), to alleviate this limitation. We demonstrate that by
carefully utilizing the widespread supervision among the image-text pairs, our
De-CLIP can learn generic visual features more efficiently. Instead of using
the single image-text contrastive supervision, we fully exploit data potential
through the use of (1) self-supervision within each modality; (2) multi-view
supervision across modalities; (3) nearest-neighbor supervision from other
similar pairs. Benefiting from intrinsic supervision, our DeCLIP-ResNet50 can
achieve 60.4% zero-shot top1 accuracy on ImageNet, which is 0.8% above the
CLIP-ResNet50 while using 7.1 x fewer data. Our DeCLIP-ResNet50 outperforms its
counterpart in 8 out of 11 visual datasets when transferred to downstream
tasks. Moreover, Scaling up the model and computing also works well in our
framework.Our code, dataset and models are released at:
this https URL

本文提出一种用于 Contrastive Language-Image Pre-training (CLIP) 的新的训练方法，Data efficient CLIP (DeCLIP)，通过充分利用图像 - 文本对之间的广泛监督来更有效地学习通用视觉特征，在使用较少数据的情况下实现 60.4% 的零样本 top1 准确率，并在转化到下游任务时优于常规 CLIP 模型。