We consider the problem of estimating an upper bound on the capacity of a
memoryless channel with unknown channel law and continuous output alphabet. A
novel data-driven algorithm is proposed that exploits the dual representation
of capacity where the maximization over the input distribution is replaced with
a minimization over a reference distribution on the channel output. To
efficiently compute the required divergence maximization between the
conditional channel and the reference distribution, we use a modified mutual
information neural estimator that takes the channel input as an additional
parameter. We numerically evaluate our approach on different memoryless
channels and show empirically that the estimated upper bounds closely converge
either to the channel capacity or to best-known lower bounds.

提出一种基于数据驱动的算法，利用最小化通道输出上的参考分布来估算未知通道法和连续输出字母表上通道容量上界的估算方法，并使用修改的互信息神经估计器来计算所需的条件通道和参考分布之间的散度最大化，在不同的无记忆通道上进行数字评估，证明了该方法估算的上界要么接近通道容量，要么接近最佳已知下界。

基于数据的容量上限估计

Data-Driven Estimation of Capacity Upper Bounds

The data-processing inequality, that is, $I(U;Y) \le I(U;X)$ for a Markov
chain $U \to X \to Y$, has been the method of choice for proving impossibility
(converse) results in information theory and many other disciplines. Various
channel-dependent improvements (called strong data-processing inequalities, or
SDPIs) of this inequality have been proposed both classically and more
recently. In this note we first survey known results relating various notions
of contraction for a single channel. Then we consider the basic extension:
given SDPI for each constituent channel in a Bayesian network, how to produce
an end-to-end SDPI?
Our approach is based on the (extract of the) Evans-Schulman method, which is
demonstrated for three different kinds of SDPIs, namely, the usual
Ahslwede-G\'acs type contraction coefficients (mutual information), Dobrushin's
contraction coefficients (total variation), and finally the $F_I$-curve (the
best possible non-linear SDPI for a given channel). Resulting bounds on the
contraction coefficients are interpreted as probability of site percolation. As
an example, we demonstrate how to obtain SDPI for an $n$-letter memoryless
channel with feedback given an SDPI for $n=1$.
Finally, we discuss a simple observation on the equivalence of a linear SDPI
and comparison to an erasure channel (in the sense of "less noisy" order). This
leads to a simple proof of a curious inequality of Samorodnitsky (2015), and
sheds light on how information spreads in the subsets of inputs of a memoryless
channel.

本文研究了数据处理不等式及其在信息论等领域中的应用，提出了一种针对贝叶斯网络的端到端强数据处理不等式构造方法，并探讨了基于线性强数据处理不等式而非耗散不等式的信息扩散方式。