Sigmoid output layers are widely used in multi-label classification (MLC)
tasks, in which multiple labels can be assigned to any input. In many practical
MLC tasks, the number of possible labels is in the thousands, often exceeding
the number of input features and resulting in a low-rank output layer. In
multi-class classification, it is known that such a low-rank output layer is a
bottleneck that can result in unargmaxable classes: classes which cannot be
predicted for any input. In this paper, we show that for MLC tasks, the
analogous sigmoid bottleneck results in exponentially many unargmaxable label
combinations. We explain how to detect these unargmaxable outputs and
demonstrate their presence in three widely used MLC datasets. We then show that
they can be prevented in practice by introducing a Discrete Fourier Transform
(DFT) output layer, which guarantees that all sparse label combinations with up
to $k$ active labels are argmaxable. Our DFT layer trains faster and is more
parameter efficient, matching the F1@k score of a sigmoid layer while using up
to 50% fewer trainable parameters. Our code is publicly available at
this https URL

在多标签分类任务中，sigmoid 输出层广泛应用；本文展示了类似 sigmoid 的瓶颈会导致指数级别的无法预测的标签组合；我们提出了一种离散傅立叶变换输出层，可以防止这种情况的发生，并且相较于 sigmoid 输出层，训练速度更快、参数利用更高效，能够在使用更少的可调参数的同时与 sigmoid 输出层达到相似的 F1 度量指标。