Deploying deep models in real-world scenarios entails a number of challenges,
including computational efficiency and real-world (e.g., long-tailed) data
distributions. We address the combined challenge of learning long-tailed
distributions using highly resource-efficient binary neural networks as
backbones. Specifically, we propose a calibrate-and-distill framework that uses
off-the-shelf pretrained full-precision models trained on balanced datasets to
use as teachers for distillation when learning binary networks on long-tailed
datasets. To better generalize to various datasets, we further propose a novel
adversarial balancing among the terms in the objective function and an
efficient multiresolution learning scheme. We conducted the largest empirical
study in the literature using 15 datasets, including newly derived long-tailed
datasets from existing balanced datasets, and show that our proposed method
outperforms prior art by large margins (>14.33% on average).

使用高度资源有效的二进制神经网络作为骨干结构来学习长尾分布，通过校准和蒸馏框架利用平衡数据集上的预训练全精度模型作为教师，结合对目标函数术语的对抗平衡和高效的多分辨率学习方案，在 15 个数据集上对最新的长尾数据集进行了最大规模的实证研究，结果显示我们的方法在平均上大幅优于现有技术（>14.33%）。