Recently, deep learning-based beamforming algorithms have shown promising
performance in target speech extraction tasks. However, most systems do not
fully utilize spatial information. In this paper, we propose a target speech
extraction network that utilizes spatial information to enhance the performance
of neural beamformer. To achieve this, we first use the UNet-TCN structure to
model input features and improve the estimation accuracy of the speech
pre-separation module by avoiding information loss caused by direct
dimensionality reduction in other models. Furthermore, we introduce a
multi-head cross-attention mechanism that enhances the neural beamformer's
perception of spatial information by making full use of the spatial information
received by the array. Experimental results demonstrate that our approach,
which incorporates a more reasonable target mask estimation network and a
spatial information-based cross-attention mechanism into the neural beamformer,
effectively improves speech separation performance.

本文提出了一种利用空间信息增强神经波束形成器性能的目标语音提取网络，该网络采用 UNet-TCN 结构模拟输入特征，并通过引入多头交叉注意力机制来利用阵列接收到的空间信息，实现了更合理的目标掩蔽估计网络和基于空间信息的交叉注意力机制，有效提高语音分离性能。

增强神经波束形成器结合空间信息用于目标语音提取

Enhanced Neural Beamformer with Spatial Information for Target Speech  Extraction

Recently, fully recurrent neural network (RNN) based end-to-end models have
been proven to be effective for multi-speaker speech recognition in both the
single-channel and multi-channel scenarios. In this work, we explore the use of
Transformer models for these tasks by focusing on two aspects. First, we
replace the RNN-based encoder-decoder in the speech recognition model with a
Transformer architecture. Second, in order to use the Transformer in the
masking network of the neural beamformer in the multi-channel case, we modify
the self-attention component to be restricted to a segment rather than the
whole sequence in order to reduce computation. Besides the model architecture
improvements, we also incorporate an external dereverberation preprocessing,
the weighted prediction error (WPE), enabling our model to handle reverberated
signals. Experiments on the spatialized wsj1-2mix corpus show that the
Transformer-based models achieve 40.9% and 25.6% relative WER reduction, down
to 12.1% and 6.4% WER, under the anechoic condition in single-channel and
multi-channel tasks, respectively, while in the reverberant case, our methods
achieve 41.5% and 13.8% relative WER reduction, down to 16.5% and 15.2% WER.

本文研究使用 Transformer 模型替代基于循环神经网络的编码器 - 解码器模型，应用于多说话者语音识别和神经束形成器中的遮盖网络，得以有效处理混响信号，并加入外部去混响预处理方法进行对比试验。实验证明，在单通道和多通道任务下，基于 Transformer 的模型相对错误率降低达 40.9% 和 25.6%，在混响环境中的相对错误率降低达 41.5% 和 13.8%。