We present a novel self-supervised approach for representation learning, particularly for the task of Visual Relationship Detection (VRD). Motivated by the effectiveness of Masked Image Modeling (MIM), we propose Masked Bounding Box Reconstruction (MBBR), a variation of MIM where a percentage of the entities/objects within a scene are masked and subsequently reconstructed based on the unmasked objects. The core idea is that, through object-level masked modeling, the network learns context-aware representations that capture the interaction of objects within a scene and thus are highly predictive of visual object relationships. We extensively evaluate learned representations, both qualitatively and quantitatively, in a few-shot setting and demonstrate the efficacy of MBBR for learning robust visual representations, particularly tailored for VRD. The proposed method is able to surpass state-of-the-art VRD methods on the Predicate Detection (PredDet) evaluation setting, using only a few annotated samples. We make our code available at https://github.com/deeplab-ai/SelfSupervisedVRD.

提出一种新颖的自监督方法用于表示学习，特别适用于视觉关系检测，通过对象级的屏蔽建模，网络学习到能够捕捉场景中对象之间相互作用的上下文感知表示方法，并在少样本情况下通过预测对象间视觉关系，实现了学习稳健视觉表示的效果优于当前最先进的视觉关系检测方法。

基于遮挡边界框重建的自监督学习视觉关系检测