In this paper, the echo state network (ESN) memory capacity, which represents
the amount of input data an ESN can store, is analyzed for a new type of deep
ESNs. In particular, two deep ESN architectures are studied. First, a parallel
deep ESN is proposed in which multiple reservoirs are connected in parallel
allowing them to average outputs of multiple ESNs, thus decreasing the
prediction error. Then, a series architecture ESN is proposed in which ESN
reservoirs are placed in cascade that the output of each ESN is the input of
the next ESN in the series. This series ESN architecture can capture more
features between the input sequence and the output sequence thus improving the
overall prediction accuracy. Fundamental analysis shows that the memory
capacity of parallel ESNs is equivalent to that of a traditional shallow ESN,
while the memory capacity of series ESNs is smaller than that of a traditional
shallow ESN.In terms of normalized root mean square error, simulation results
show that the parallel deep ESN achieves 38.5% reduction compared to the
traditional shallow ESN while the series deep ESN achieves 16.8% reduction.

本文研究了一种新型深度 ESN，包括并行深度 ESN 和级联深度 ESN，分析了它们的存储容量和预测准确度，并发现并行深度 ESN 比传统浅层 ESN 实现了更好的预测效果，而级联深度 ESN 存储容量小于传统浅层 ESN，但是能提高整体预测精度。