Image representations are commonly learned from class labels, which are a
simplistic approximation of human image understanding. In this paper we
demonstrate that transferable representations of images can be learned without
manual annotations by modeling human visual attention. The basis of our
analyses is a unique gaze tracking dataset of sonographers performing routine
clinical fetal anomaly screenings. Models of sonographer visual attention are
learned by training a convolutional neural network (CNN) to predict gaze on
ultrasound video frames through visual saliency prediction or gaze-point
regression. We evaluate the transferability of the learned representations to
the task of ultrasound standard plane detection in two contexts. Firstly, we
perform transfer learning by fine-tuning the CNN with a limited number of
labeled standard plane images. We find that fine-tuning the saliency predictor
is superior to training from random initialization, with an average F1-score
improvement of 9.6% overall and 15.3% for the cardiac planes. Secondly, we
train a simple softmax regression on the feature activations of each CNN layer
in order to evaluate the representations independently of transfer learning
hyper-parameters. We find that the attention models derive strong
representations, approaching the precision of a fully-supervised baseline model
for all but the last layer.

本文研究通过建立基于人类视觉注意的模型来学习图像的可转移表示，以取代直接学习类标签所造成的图像理解的简化。通过在超声视频中预测视觉显著性或注视点回归，利用卷积神经网络从注视数据集中学习视觉注意模型，并评估该模型在超声标准平面检测任务中的可迁移性。实验表明，在有限标记的场景下，微调视觉显著性预测器优于随机初始化，平均 F1 分数整体提高 9.6％，对于心脏平面提高了 15.3％。另外，该学习方法生成的表示发现是可靠的，在近乎监督的基准模型的精度上接近。