Sparse deep learning has become a popular technique for improving the
performance of deep neural networks in areas such as uncertainty
quantification, variable selection, and large-scale network compression.
However, most existing research has focused on problems where the observations
are independent and identically distributed (i.i.d.), and there has been little
work on the problems where the observations are dependent, such as time series
data and sequential data in natural language processing. This paper aims to
address this gap by studying the theory for sparse deep learning with dependent
data. We show that sparse recurrent neural networks (RNNs) can be consistently
estimated, and their predictions are asymptotically normally distributed under
appropriate assumptions, enabling the prediction uncertainty to be correctly
quantified. Our numerical results show that sparse deep learning outperforms
state-of-the-art methods, such as conformal predictions, in prediction
uncertainty quantification for time series data. Furthermore, our results
indicate that the proposed method can consistently identify the autoregressive
order for time series data and outperform existing methods in large-scale model
compression. Our proposed method has important practical implications in fields
such as finance, healthcare, and energy, where both accurate point estimates
and prediction uncertainty quantification are of concern.

稀疏深度学习在处理依赖数据方面的预测不确定性量化、顶点估计和模型压缩等领域中已成为一种流行的技术。本文通过研究稀疏深度学习与依赖数据的理论，表明稀疏循环神经网络能够一致地估计，并且在适当假设下，其预测呈渐近正态分布，从而能够正确量化预测的不确定性。数值实验结果显示，稀疏深度学习在时间序列数据的预测不确定性量化方面优于诸如互联置信度等先进方法。此外，我们的研究结果还表明，所提出的方法能够一致地识别时间序列数据的自回归阶数，并在大规模模型压缩方面优于现有方法。该方法在金融、医疗和能源等领域具有重要的实际意义，因为准确的顶点估计和预测不确定性量化都是关注的焦点。