In this paper we consider the problem of multi-task learning, in which a learner is given a collection of prediction tasks that need to be solved. In contrast to previous work, we give up on the assumption that labeled training data is available for all tasks. Instead, we propose an