In some practical learning tasks, such as traffic video analysis, the number
of available training samples is restricted by different factors, such as
limited communication bandwidth and computation power; therefore, it is
imperative to select diverse data samples that contribute the most to the
quality of the learning system. One popular approach to selecting diverse
samples is Determinantal Point Process (DPP). However, it suffers from a few
known drawbacks, such as restriction of the number of samples to the rank of
the similarity matrix, and not being customizable for specific learning tasks
(e.g., multi-level classification tasks). In this paper, we propose a new way
of measuring task-oriented diversity based on the Rate-Distortion (RD) theory,
appropriate for multi-level classification. To this end, we establish a
fundamental relationship between DPP and RD theory, which led to designing
RD-DPP, an RD-based value function to evaluate the diversity gain of data
samples. We also observe that the upper bound of the diversity of data selected
by DPP has a universal trend of phase transition that quickly approaches its
maximum point, then slowly converges to its final limits, meaning that DPP is
beneficial only at the beginning of sample accumulation. We use this fact to
design a bi-modal approach for sequential data selection.

该研究提出了一种基于速率失真理论的选择具有任务导向的多级分类数据样本的新方法，称为 RD-DPP，可用于评估数据样本的情境感知多样性。此外，该研究观察到了基于确定性点过程的数据选择在样本累积过程中存在相变现象，从而设计出一种双峰方法以获得更大的多样性收益。

RD-DPP: 速率失真理论 meets 确定性点过程以实现多样化学习数据样本

RD-DPP: Rate-Distortion Theory Meets Determinantal Point Process to  Diversify Learning Data Samples

The most important obstacles facing multi-document summarization include
excessive redundancy in source descriptions and the looming shortage of
training data. These obstacles prevent encoder-decoder models from being used
directly, but optimization-based methods such as determinantal point processes
(DPPs) are known to handle them well. In this paper we seek to strengthen a
DPP-based method for extractive multi-document summarization by presenting a
novel similarity measure inspired by capsule networks. The approach measures
redundancy between a pair of sentences based on surface form and semantic
information. We show that our DPP system with improved similarity measure
performs competitively, outperforming strong summarization baselines on
benchmark datasets. Our findings are particularly meaningful for summarizing
documents created by multiple authors containing redundant yet lexically
diverse expressions.

本研究旨在通过引入胶囊网络的相似度测量来加强基于 DPP 的多文档摘要方法，以解决大量冗余信息和训练数据短缺等难题。我们使用基于表面形式和语义信息的新相似度测量方法来评估句子对之间的冗余程度，并表明我们改进后的 DPP 系统具有更为出色的表现，超越了基准数据集上的强大摘要基线。