Deep learning faces significant challenges during the training of neural
networks, including internal covariate shift, label shift, vanishing/exploding
gradients, overfitting, and computational complexity. While conventional
normalization methods, such as Batch Normalization, aim to tackle some of these
issues, they often depend on assumptions that constrain their adaptability.
Mixture Normalization faces computational hurdles in its pursuit of handling
multiple Gaussian distributions. This paper introduces Cluster-Based
Normalization (CB-Norm) in two variants - Supervised Cluster-Based
Normalization (SCB-Norm) and Unsupervised Cluster-Based Normalization
(UCB-Norm) - proposing a groundbreaking one-step normalization approach.
CB-Norm leverages a Gaussian mixture model to specifically address challenges
related to gradient stability and learning acceleration. For SCB-Norm, a
supervised variant, the novel mechanism involves introducing predefined data
partitioning, termed clusters, to normalize activations based on the assigned
cluster. This cluster-driven approach creates a space that conforms to a
Gaussian mixture model. On the other hand, UCB-Norm, an unsupervised
counterpart, dynamically clusters neuron activations during training, adapting
to task-specific challenges without relying on predefined data partitions
(clusters). This dual approach ensures flexibility in addressing diverse
learning scenarios. CB-Norm innovatively uses a one-step normalization
approach, where parameters of each mixture component (cluster in activation
space) serve as weights for deep neural networks. This adaptive clustering
process tackles both clustering and resolution of deep neural network tasks
concurrently during training, signifying a notable advancement in the field.

深度学习面临的挑战包括内部协变量偏移、标签偏移、梯度消失 / 爆炸、过拟合和计算复杂性。本文引入基于聚类的规范化方法 (CB-Norm) 的两个变种 —— 有监督聚类规范化 (SCB-Norm) 和无监督聚类规范化 (UCB-Norm)，提出了一种突破性的一步规范化方法。CB-Norm 利用高斯混合模型来解决与梯度稳定性和学习加速相关的挑战。SCB-Norm 引入了预定义的数据分区，通过分配的聚类来规范化激活。UCB-Norm 动态地对神经元激活进行聚类，适应任务特定的挑战，而不依赖预定义的数据分区。这种双重方法确保了对多样化学习场景的灵活应对。CB-Norm 创新地采用一步规范化的方法，其中每个混合成分的参数（激活空间中的聚类）作为深度神经网络的权重。这种自适应聚类过程在训练过程中同时解决了聚类和深度神经网络任务的分辨率，标志着该领域的一个显著进步。

基于聚类的神经网络标准化层

Cluster-Based Normalization Layer for Neural Networks

Providing Reinforcement Learning (RL) agents with human feedback can
dramatically improve various aspects of learning. However, previous methods
require human observer to give inputs explicitly (e.g., press buttons, voice
interface), burdening the human in the loop of RL agent's learning process.
Further, it is sometimes difficult or impossible to obtain the explicit human
advise (feedback), e.g., autonomous driving, disabled rehabilitation, etc. In
this work, we investigate capturing human's intrinsic reactions as implicit
(and natural) feedback through EEG in the form of error-related potentials
(ErrP), providing a natural and direct way for humans to improve the RL agent
learning. As such, the human intelligence can be integrated via implicit
feedback with RL algorithms to accelerate the learning of RL agent. We develop
three reasonably complex 2D discrete navigational games to experimentally
evaluate the overall performance of the proposed work. Major contributions of
our work are as follows,
(i) we propose and experimentally validate the zero-shot learning of ErrPs,
where the ErrPs can be learned for one game, and transferred to other unseen
games, (ii) we propose a novel RL framework for integrating implicit human
feedbacks via ErrPs with RL agent, improving the label efficiency and
robustness to human mistakes, and (iii) compared to prior works, we scale the
application of ErrPs to reasonably complex environments, and demonstrate the
significance of our approach for accelerated learning through real user
experiments.

该研究提出通过 EEG 作为自然反应的形式来捕获人类的内在反应作为隐式反馈，并将人类智能通过隐式反馈与 RL 算法相结合，以加速 RL 智能体的学习。