We suggest a simple Gaussian mixture model for data generation that complies
with Feldman's long tail theory (2020). We demonstrate that a linear classifier
cannot decrease the generalization error below a certain level in the proposed
model, whereas a nonlinear classifier with a memorization capacity can. This
confirms that for long-tailed distributions, rare training examples must be
considered for optimal generalization to new data. Finally, we show that the
performance gap between linear and nonlinear models can be lessened as the tail
becomes shorter in the subpopulation frequency distribution, as confirmed by
experiments on synthetic and real data.

我们提出了一个简单的高斯混合模型用于数据生成，符合 Feldman 的长尾理论（2020）。我们证明了在线性分类器中，在所提出的模型中无法将泛化误差降低到一定水平以下，而具有记忆能力的非线性分类器可以做到。这证实了对于长尾分布，必须考虑到罕见的训练样本才能实现对新数据的最佳泛化。最后，我们展示了在线性和非线性模型之间的性能差距可以在亚种群频率分布的尾部变短时减小，这在合成和实际数据的实验中得到了证实。