We discovered that the Deep ReLU networks (or Multilayer Perceptron
architecture) demonstrate an 'over-generalization' phenomenon. That is, the
output values for the inputs that were not seen during training are mapped
close to the output range that were observed during the learning process. In
other words, the MLP learns a many-to-one mapping and this effect is more
prominent as we increase the number of layers or depth of the MLP. We utilize
this property of Deep ReLU networks to design a dataset kaleidoscope, termed as
'Generative Kaleidoscopic Networks'. Briefly, if we learn a MLP to map from
input $x\in\mathbb{R}^D$ to itself $f_\mathcal{N}(x)\rightarrow x$, the
'Kaleidoscopic sampling' procedure starts with a random input noise
$z\in\mathbb{R}^D$ and recursively applies $f_\mathcal{N}(\cdots
f_\mathcal{N}(z)\cdots )$. After a burn-in period duration, we start observing
samples from the input distribution and we found that deeper the MLP, higher is
the quality of samples recovered. Scope: We observed this phenomenon to various
degrees for the other deep learning architectures like CNNs, Transformers &
U-Nets and we are currently investigating them further.

发现深度 ReLU 网络（或多层感知器架构）表现出 ' 过度泛化 ' 现象并利用这一特性设计了数据集炫斑网络，称为 ' 生成炫斑网络 '。此现象在其他深度学习架构（如 CNNs、Transformers 和 U-Nets）中也观察到，并正在进一步研究。

生成万花筒网络

Generative Kaleidoscopic Networks

Over-generalization is a thorny issue in cognitive science, where people may
become overly cautious due to past experiences. Agents in multi-agent
reinforcement learning (MARL) also have been found to suffer relative
over-generalization (RO) as people do and stuck to sub-optimal cooperation.
Recent methods have shown that assigning reasoning ability to agents can
mitigate RO algorithmically and empirically, but there has been a lack of
theoretical understanding of RO, let alone designing provably RO-free methods.
This paper first proves that RO can be avoided when the MARL method satisfies a
consistent reasoning requirement under certain conditions. Then we introduce a
novel reasoning framework, called negotiated reasoning, that first builds the
connection between reasoning and RO with theoretical justifications. After
that, we propose an instantiated algorithm, Stein variational negotiated
reasoning (SVNR), which uses Stein variational gradient descent to derive a
negotiation policy that provably avoids RO in MARL under maximum entropy policy
iteration. The method is further parameterized with neural networks for
amortized learning, making computation efficient. Numerical experiments on many
RO-challenged environments demonstrate the superiority and efficiency of SVNR
compared to state-of-the-art methods in addressing RO.

本文提出了一种新的协商推理框架和实例化算法 Stein 变分协商推理 (SVNR)，其使用 Stein 变分梯度下降来导出一种协商政策，以便在最大熵策略迭代下在多智能体强化学习中避免 RO 。对许多 RO 挑战环境的数值实验证明了 SVNR 在解决 RO 中的卓越性和效率。