Although neural networks are conventionally optimized towards zero training
loss, it has been recently learned that targeting a non-zero training loss
threshold, referred to as a flood level, often enables better test time
generalization. Current approaches, however, apply the same constant flood
level to all training samples, which inherently assumes all the samples have
the same difficulty. We present AdaFlood, a novel flood regularization method
that adapts the flood level of each training sample according to the difficulty
of the sample. Intuitively, since training samples are not equal in difficulty,
the target training loss should be conditioned on the instance. Experiments on
datasets covering four diverse input modalities - text, images, asynchronous
event sequences, and tabular - demonstrate the versatility of AdaFlood across
data domains and noise levels.

传统上，神经网络会优化目标函数使得训练损失为零，但最近研究发现设定一个非零训练损失阈值 (即洪水水位) 通常能够实现更好的测试泛化。然而，现有的方法将相同的恒定洪水水位应用于所有训练样本，这在本质上假设所有样本具有相同的难度。本文提出了 AdaFlood，一种新颖的洪水正则化方法，根据样本的难度调整每个训练样本的洪水水位。直观来说，由于训练样本的难度不同，目标训练损失应该与实例有关系。在涵盖文本、图像、异步事件序列和表格等四种不同输入模态的数据集上的实验证明了 AdaFlood 在不同数据领域和噪声水平下的多功能性。