In a multi-channel separation task with multiple speakers, we aim to recover
all individual speech signals from the mixture. In contrast to single-channel
approaches, which rely on the different spectro-temporal characteristics of the
speech signals, multi-channel approaches should additionally utilize the
different spatial locations of the sources for a more powerful separation
especially when the number of sources increases. To enhance the spatial
processing in a multi-channel source separation scenario, in this work, we
propose a deep neural network (DNN) based spatially selective filter (SSF) that
can be spatially steered to extract the speaker of interest by initializing a
recurrent neural network layer with the target direction. We compare the
proposed SSF with a common end-to-end direct separation (DS) approach trained
using utterance-wise permutation invariant training (PIT), which only
implicitly learns to perform spatial filtering. We show that the SSF has a
clear advantage over a DS approach with the same underlying network
architecture when there are more than two speakers in the mixture, which can be
attributed to a better use of the spatial information. Furthermore, we find
that the SSF generalizes much better to additional noise sources that were not
seen during training.

本文中我们提出了基于深度神经网络和循环神经网络的空间选择性滤波器 (SSF)，在多扬声器多声道分离任务中，可以从混合信号中提取出特定说话者的声音信息，并与直接分离方法进行了比较。结果表明，SSF 具有更好的空间信息利用和泛化能力。

利用空间选择性深度非线性滤波器的多通道语音分离

Multi-channel Speech Separation Using Spatially Selective Deep  Non-linear Filters

Self-attention is powerful in modeling long-range dependencies, but it is
weak in local finer-level feature learning. The performance of local
self-attention (LSA) is just on par with convolution and inferior to dynamic
filters, which puzzles researchers on whether to use LSA or its counterparts,
which one is better, and what makes LSA mediocre. To clarify these, we
comprehensively investigate LSA and its counterparts from two sides:
\emph{channel setting} and \emph{spatial processing}. We find that the devil
lies in the generation and application of spatial attention, where relative
position embeddings and the neighboring filter application are key factors.
Based on these findings, we propose the enhanced local self-attention (ELSA)
with Hadamard attention and the ghost head. Hadamard attention introduces the
Hadamard product to efficiently generate attention in the neighboring case,
while maintaining the high-order mapping. The ghost head combines attention
maps with static matrices to increase channel capacity. Experiments demonstrate
the effectiveness of ELSA. Without architecture / hyperparameter modification,
drop-in replacing LSA with ELSA boosts Swin Transformer \cite{swin} by up to
+1.4 on top-1 accuracy. ELSA also consistently benefits VOLO \cite{volo} from
D1 to D5, where ELSA-VOLO-D5 achieves 87.2 on the ImageNet-1K without extra
training images. In addition, we evaluate ELSA in downstream tasks. ELSA
significantly improves the baseline by up to +1.9 box Ap / +1.3 mask Ap on the
COCO, and by up to +1.9 mIoU on the ADE20K. Code is available at
https://github.com/damo-cv/ELSA.

本文通过从通道设置和空间处理两个方面综合调查了本地自注意力和其同行的性能，发现空间注意力的生成和应用是影响性能的关键因素，提出了增强型本地自注意力（ELSA）并通过实验验证其有效性。