We study the approximation capacity of deep ReLU recurrent neural networks (RNNs) and explore the convergence properties of nonparametric least squares regression using RNNs. We derive upper bounds on the approximation error of RNNs for H\"older smooth functions, in the sense that the output at each time step of an RNN can approximate a H\"older function that depends only on past and current information, termed a past-dependent function. This allows a carefully constructed RNN to simultaneously approximate a sequence of past-dependent H\"older functions. We apply these approximation results to derive non-asymptotic upper bounds for the prediction error of the empirical risk minimizer in regression problem. Our error bounds achieve minimax optimal rate under both exponentially $\beta$-mixing and i.i.d. data assumptions, improving upon existing ones. Our results provide statistical guarantees on the performance of RNNs.

本研究解决了深度ReLU递归神经网络（RNN）的逼近能力及其在非参数最小二乘回归中的收敛性问题。通过对Hölder光滑函数的逼近误差进行界定，提出了一种新颖的RNN构建方法，能够优化回归问题中的预测误差界限，实现了在数据假设下的最优收敛速率。这一结果为RNN的性能提供了统计保证。

递归神经网络的逼近界限及其在回归中的应用