Images suffer from heavy spatial redundancy because pixels in neighboring
regions are spatially correlated. Existing approaches strive to overcome this
limitation by reducing less meaningful image regions. However, current leading
methods rely on supervisory signals. They may compel models to preserve content
that aligns with labeled categories and discard content belonging to unlabeled
categories. This categorical inductive bias makes these methods less effective
in real-world scenarios. To address this issue, we propose a self-supervised
framework for image redundancy reduction called Learning to Rank Patches
(LTRP). We observe that image reconstruction of masked image modeling models is
sensitive to the removal of visible patches when the masking ratio is high
(e.g., 90\%). Building upon it, we implement LTRP via two steps: inferring the
semantic density score of each patch by quantifying variation between
reconstructions with and without this patch, and learning to rank the patches
with the pseudo score. The entire process is self-supervised, thus getting out
of the dilemma of categorical inductive bias. We design extensive experiments
on different datasets and tasks. The results demonstrate that LTRP outperforms
both supervised and other self-supervised methods due to the fair assessment of
image content.

我们提出了一种自我监督的图像冗余减少框架，称为学习排名补丁（LTRP），它通过推断每个补丁的语义密度得分，并学习使用伪分数对补丁进行排序，充分评估图像内容，克服了分类归纳偏差的困境，从而在不同数据集和任务上优于监督和其他自我监督方法。