Learning from Label Proportions (LLP) is a learning problem where only
aggregate level labels are available for groups of instances, called bags,
during training, and the aim is to get the best performance at the
instance-level on the test data. This setting arises in domains like
advertising and medicine due to privacy considerations. We propose a novel
algorithmic framework for this problem that iteratively performs two main
steps. For the first step (Pseudo Labeling) in every iteration, we define a
Gibbs distribution over binary instance labels that incorporates a) covariate
information through the constraint that instances with similar covariates
should have similar labels and b) the bag level aggregated label. We then use
Belief Propagation (BP) to marginalize the Gibbs distribution to obtain pseudo
labels. In the second step (Embedding Refinement), we use the pseudo labels to
provide supervision for a learner that yields a better embedding. Further, we
iterate on the two steps again by using the second step's embeddings as new
covariates for the next iteration. In the final iteration, a classifier is
trained using the pseudo labels. Our algorithm displays strong gains against
several SOTA baselines (up to 15%) for the LLP Binary Classification problem on
various dataset types - tabular and Image. We achieve these improvements with
minimal computational overhead above standard supervised learning due to Belief
Propagation, for large bag sizes, even for a million samples.

通过学习标签比例 (LLP)，我们提出了一种新的算法框架，其中迭代地执行伪标签和嵌入改进两个主要步骤，利用 Gibbs 分布和 Belief Propagation，在不牺牲计算效率的情况下，取得了标签比例学习问题上的显著改进。