Reliable detection of the prodromal stages of Alzheimer's disease (AD)
remains difficult even today because, unlike other neurocognitive impairments,
there is no definitive diagnosis of AD in vivo. In this context, existing
research has shown that patients often develop language impairment even in mild
AD conditions. We propose a multimodal deep learning method that utilizes
speech and the corresponding transcript simultaneously to detect AD. For audio
signals, the proposed audio-based network, a convolutional neural network (CNN)
based model, predicts the diagnosis for multiple speech segments, which are
combined for the final prediction. Similarly, we use contextual embedding
extracted from BERT concatenated with a CNN-generated embedding for classifying
the transcript. The individual predictions of the two models are then combined
to make the final classification. We also perform experiments to analyze the
model performance when Automated Speech Recognition (ASR) system generated
transcripts are used instead of manual transcription in the text-based model.
The proposed method achieves 85.3% 10-fold cross-validation accuracy when
trained and evaluated on the Dementiabank Pitt corpus.

使用多模式深度学习方法通过语音和相应的文本同时检测多种言语段落的诊断，其模型在 Dementiabank Pitt 语料库上获得了 85.3% 的准确率。

从语音和文本多模态检测阿尔茨海默病

Multi-Modal Detection of Alzheimer's Disease from Speech and Text

Dysarthria is a motor speech impairment affecting millions of people.
Dysarthric speech can be far less intelligible than those of non-dysarthric
speakers, causing significant communication difficulties. The goal of our work
is to develop a model for dysarthric to healthy speech conversion using
Cycle-consistent GAN. Using 18,700 dysarthric and 8,610 healthy control Korean
utterances that were recorded for the purpose of automatic recognition of voice
keyboard in a previous study, the generator is trained to transform dysarthric
to healthy speech in the spectral domain, which is then converted back to
speech. Objective evaluation using automatic speech recognition of the
generated utterance on a held-out test set shows that the recognition
performance is improved compared with the original dysarthic speech after
performing adversarial training, as the absolute WER has been lowered by 33.4%.
It demonstrates that the proposed GAN-based conversion method is useful for
improving dysarthric speech intelligibility.

本研究旨在使用循环一致性生成对抗网络模型将口吃音转换为健康人的语音，使用 18,700 个口吃音和 8,610 个健康对照韩语发音训练了本项目的发生器，进而证明了该基于 GAN 的转换方法对于改善口吃音的可懂度是有用的。