What makes waveform-based deep learning so hard? Despite numerous attempts at
training convolutional neural networks (convnets) for filterbank design, they
often fail to outperform hand-crafted baselines. This is all the more
surprising because these baselines are linear time-invariant systems: as such,
their transfer functions could be accurately represented by a convnet with a
large receptive field. In this article, we elaborate on the statistical
properties of simple convnets from the mathematical perspective of random
convolutional operators. We find that FIR filterbanks with random Gaussian
weights are ill-conditioned for large filters and locally periodic input
signals, which both are typical in audio signal processing applications.
Furthermore, we observe that expected energy preservation of a random
filterbank is not sufficient for numerical stability and derive theoretical
bounds for its expected frame bounds.

波形为基础的深度学习为何如此困难？尽管有很多尝试去训练卷积神经网络 (convnets) 进行滤波器设计，但它们通常无法胜过手工制作的基准线。本文从随机卷积算子的数学角度详细论述了简单卷积神经网络的统计性质。我们发现，具有随机高斯权重的有限脉冲响应 (FIR) 滤波器对于大滤波器和局部周期性输入信号是病态的，而这两者在音频信号处理应用中很常见。此外，我们观察到随机滤波器组的期望能量保持并不足以确保数值稳定性，并为其期望帧界导出了理论上的限制。

随机滤波器组的能量保持和稳定性

Energy Preservation and Stability of Random Filterbanks

Waveform-based deep learning faces a dilemma between nonparametric and
parametric approaches. On one hand, convolutional neural networks (convnets)
may approximate any linear time-invariant system; yet, in practice, their
frequency responses become more irregular as their receptive fields grow. On
the other hand, a parametric model such as LEAF is guaranteed to yield Gabor
filters, hence an optimal time-frequency localization; yet, this strong
inductive bias comes at the detriment of representational capacity. In this
paper, we aim to overcome this dilemma by introducing a neural audio model,
named multiresolution neural network (MuReNN). The key idea behind MuReNN is to
train separate convolutional operators over the octave subbands of a discrete
wavelet transform (DWT). Since the scale of DWT atoms grows exponentially
between octaves, the receptive fields of the subsequent learnable convolutions
in MuReNN are dilated accordingly. For a given real-world dataset, we fit the
magnitude response of MuReNN to that of a well-established auditory filterbank:
Gammatone for speech, CQT for music, and third-octave for urban sounds,
respectively. This is a form of knowledge distillation (KD), in which the
filterbank ''teacher'' is engineered by domain knowledge while the neural
network ''student'' is optimized from data. We compare MuReNN to the state of
the art in terms of goodness of fit after KD on a hold-out set and in terms of
Heisenberg time-frequency localization. Compared to convnets and Gabor
convolutions, we find that MuReNN reaches state-of-the-art performance on all
three optimization problems.

通过引入名为 MuReNN 的神经音频模型，我们旨在解决深度学习在波形处理中的非参数和参数方法之间的矛盾，MuReNN 通过训练离散小波变换 (DWT) 的八度子带上的独立卷积运算器来拓宽其感受野，将声音数据集的幅度响应与 Gammatone、CQT 和三分之一八度声音滤波器进行知识蒸馏，并在保留集上与现有的方法进行对比，结果表明 MuReNN 在所有三个优化问题上达到了最先进的性能。