In a multi-speaker "cocktail party" scenario, a listener can selectively
attend to a speaker of interest. Studies into the human auditory attention
network demonstrate cortical entrainment to speech envelopes resulting in
highly correlated Electroencephalography (EEG) measurements. Current trends in
EEG-based auditory attention detection (AAD) using artificial neural networks
(ANN) are not practical for edge-computing platforms due to longer decision
windows using several EEG channels, with higher power consumption and larger
memory footprint requirements. Nor are ANNs capable of accurately modeling the
brain's top-down attention network since the cortical organization is complex
and layer. In this paper, we propose a hybrid convolutional neural
network-spiking neural network (CNN-SNN) corticomorphic architecture, inspired
by the auditory cortex, which uses EEG data along with multi-speaker speech
envelopes to successfully decode auditory attention with low latency down to 1
second, using only 8 EEG electrodes strategically placed close to the auditory
cortex, at a significantly higher accuracy of 91.03%, compared to the
state-of-the-art. Simultaneously, when compared to a traditional CNN reference
model, our model uses ~15% fewer parameters at a lower bit precision resulting
in ~57% memory footprint reduction. The results show great promise for
edge-computing in brain-embedded devices, like smart hearing aids.

本文提出了一种受听觉皮层启发的混合卷积神经网络 - 脉冲神经网络（CNN-SNN）器件，使用 EEG 数据及多说者语音包络成功计算出空间定位，并取得了 91.03% 的高精度，同时相较于传统方法，参数使用降低了 15%，存储空间降低了 57%，并具有较大的应用潜力，特别推荐应用于智能助听器等装置上。

基于 EEG 的低占用和低延迟听觉注意力检测的 Corticomorphic 混合 CNN-SNN 结构

Corticomorphic Hybrid CNN-SNN Architecture for EEG-based Low-footprint  Low-latency Auditory Attention Detection

Multi-resolution spectro-temporal features of a speech signal represent how
the brain perceives sounds by tuning cortical cells to different spectral and
temporal modulations. These features produce a higher dimensional
representation of the speech signals. The purpose of this paper is to evaluate
how well the auditory cortex representation of speech signals contribute to
estimate articulatory features of those corresponding signals. Since obtaining
articulatory features from acoustic features of speech signals has been a
challenging topic of interest for different speech communities, we investigate
the possibility of using this multi-resolution representation of speech signals
as acoustic features. We used U. of Wisconsin X-ray Microbeam (XRMB) database
of clean speech signals to train a feed-forward deep neural network (DNN) to
estimate articulatory trajectories of six tract variables. The optimal set of
multi-resolution spectro-temporal features to train the model were chosen using
appropriate scale and rate vector parameters to obtain the best performing
model. Experiments achieved a correlation of 0.675 with ground-truth tract
variables. We compared the performance of this speech inversion system with
prior experiments conducted using Mel Frequency Cepstral Coefficients (MFCCs).

本研究评估了利用多分辨率频谱时间特征作为声学特征对语音信号进行表达，从而从听觉皮层表征语音信号的好处，以推测相应信号的发音学特征。研究使用了威斯康辛大学 X 光微束（XRMB）数据库的语音信号来训练前馈深度神经网络（DNN）以估计 6 个道路变量的发音轨迹。实验结果表明，该方法与利用 Mel 频率倒谱系数的先前实验相比，能达到更高的相关性（0.675）。

使用多分辨率声谱时序表示的声学到口腔运动的语音反演

Acoustic To Articulatory Speech Inversion Using Multi-Resolution Spectro-Temporal Representations Of Speech Signals

Representations in the auditory cortex might be based on mechanisms similar
to the visual ventral stream; modules for building invariance to
transformations and multiple layers for compositionality and selectivity. In
this paper we propose the use of such computational modules for extracting
invariant and discriminative audio representations. Building on a theory of
invariance in hierarchical architectures, we propose a novel, mid-level
representation for acoustical signals, using the empirical distributions of
projections on a set of templates and their transformations. Under the
assumption that, by construction, this dictionary of templates is composed from
similar classes, and samples the orbit of variance-inducing signal
transformations (such as shift and scale), the resulting signature is
theoretically guaranteed to be unique, invariant to transformations and stable
to deformations. Modules of projection and pooling can then constitute layers
of deep networks, for learning composite representations. We present the main
theoretical and computational aspects of a framework for unsupervised learning
of invariant audio representations, empirically evaluated on music genre
classification.

本研究提出使用视觉中腹流的机制，构建计算模块以提取不变和有差别的音频表示，实现音频流派分类的无监督学习