Machine learning based approaches are being increasingly used for designing
decoders for next generation communication systems. One widely used framework
is neural belief propagation (NBP), which unfolds the belief propagation (BP)
iterations into a deep neural network and the parameters are trained in a
data-driven manner. NBP decoders have been shown to improve upon classical
decoding algorithms. In this paper, we investigate the generalization
capabilities of NBP decoders. Specifically, the generalization gap of a decoder
is the difference between empirical and expected bit-error-rate(s). We present
new theoretical results which bound this gap and show the dependence on the
decoder complexity, in terms of code parameters (blocklength, message length,
variable/check node degrees), decoding iterations, and the training dataset
size. Results are presented for both regular and irregular parity-check
matrices. To the best of our knowledge, this is the first set of theoretical
results on generalization performance of neural network based decoders. We
present experimental results to show the dependence of generalization gap on
the training dataset size, and decoding iterations for different codes.

本文研究神经信念传播解码器的泛化能力，利用机器学习方法设计下一代通信系统的译码器，提高传统解码算法的性能。通过理论与实验结果给出了解码器的泛化缺口，以及其与代码参数、解码迭代次数、训练数据集大小等的关系。

神经置信传播译码器的泛化界限

Generalization Bounds for Neural Belief Propagation Decoders

While state-of-the-art neural network models continue to achieve lower
perplexity scores on language modeling benchmarks, it remains unknown whether
optimizing for broad-coverage predictive performance leads to human-like
syntactic knowledge. Furthermore, existing work has not provided a clear
picture about the model properties required to produce proper syntactic
generalizations. We present a systematic evaluation of the syntactic knowledge
of neural language models, testing 20 combinations of model types and data
sizes on a set of 34 English-language syntactic test suites. We find
substantial differences in syntactic generalization performance by model
architecture, with sequential models underperforming other architectures.
Factorially manipulating model architecture and training dataset size (1M--40M
words), we find that variability in syntactic generalization performance is
substantially greater by architecture than by dataset size for the corpora
tested in our experiments. Our results also reveal a dissociation between
perplexity and syntactic generalization performance.

我们对神经语言模型的句法知识进行了系统评估，发现模型架构的不同对句法泛化性能产生了显著影响，而不同数据集大小的影响相对较低。