In this paper, we explore methods of complicating self-supervised tasks for representation learning. That is, we do severe damage to data and encourage a network to recover them. First, we complicate each of three powerful self-supervised task candidates: jigsaw puzzle, inpainting, and colorization. In addition, we introduce a novel complicated self-supervised task called "Completing damaged jigsaw puzzles" which is puzzles with one piece missing and the other pieces without color. We train a convolutional neural network not only to solve the puzzles, but also generate the missing content and colorize the puzzles. The recovery of the aforementioned damage pushes the network to obtain robust and general-purpose representations. We demonstrate that complicating the self-supervised tasks improves their original versions and that our final task learns more robust and transferable representations compared to the previous methods, as well as the simple combination of our candidate tasks. Our approach achieves state-of-the-art performance in transfer learning on PASCAL classification and semantic segmentation.

本文探索用复杂的自监督任务进行表示学习的方法，介绍了一个新的复杂自监督任务“完善受损拼图”，使用卷积神经网络训练，证明使用复杂的自监督任务有助于学习更健壮和通用的表示，证明这种方法在PASCAL分类和语义分割的迁移学习中实现了最先进的性能。

通过完成损坏的拼图学习图像表示