We present two multimodal fusion-based deep learning models that consume ASR
transcribed speech and acoustic data simultaneously to classify whether a
speaker in a structured diagnostic task has Alzheimer's Disease and to what
degree, evaluating the ADReSSo challenge 2021 data. Our best model, a BiLSTM
with highway layers using words, word probabilities, disfluency features, pause
information, and a variety of acoustic features, achieves an accuracy of 84%
and RSME error prediction of 4.26 on MMSE cognitive scores. While predicting
cognitive decline is more challenging, our models show improvement using the
multimodal approach and word probabilities, disfluency and pause information
over word-only models. We show considerable gains for AD classification using
multimodal fusion and gating, which can effectively deal with noisy inputs from
acoustic features and ASR hypotheses.

本文介绍了两种基于多模态融合的深度学习模型，同时使用 ASR 转录的语音和声纹数据，对结构化诊断任务中的说话人是否存在阿尔茨海默病进行分类，并评估了 ADReSSo 挑战 2021 数据，其中最佳模型 BiLSTM 采用了包括单词、单词概率、口吃特征、停顿信息和各种声学特征的 highway 层，实现了 84％ 的准确率和 4.26 的 MMSE 认知分数预测 RSME 误差。通过使用多模态方法和单词概率、口吃和停顿信息，我们的模型在预测认知衰退时表现出了提高，并且对于使用多模态融合和门控技术进行的 AD 分类，我们展示了相当大的收益。

使用听觉、词汇、语调和停顿特征对阿尔茨海默病痴呆进行识别并抗干扰

Alzheimer's Dementia Recognition Using Acoustic, Lexical, Disfluency and  Speech Pause Features Robust to Noisy Inputs

Attribution methods are an easy to use tool for investigating and validating
machine learning models. Multiple methods have been suggested in the literature
and it is not yet clear which method is most suitable for a given task. In this
study, we tested the robustness of four attribution methods, namely
gradient*input, guided backpropagation, layer-wise relevance propagation and
occlusion, for the task of Alzheimer's disease classification. We have
repeatedly trained a convolutional neural network (CNN) with identical training
settings in order to separate structural MRI data of patients with Alzheimer's
disease and healthy controls. Afterwards, we produced attribution maps for each
subject in the test data and quantitatively compared them across models and
attribution methods. We show that visual comparison is not sufficient and that
some widely used attribution methods produce highly inconsistent outcomes.

研究了四种归因方法在阿尔茨海默病分类任务中的有效性，并发现某些广泛使用的归因方法产生高度不一致的结果。