This paper seeks to address the dense labeling problems where a significant
fraction of the dataset can be pruned without sacrificing much accuracy. We
observe that, on standard medical image segmentation benchmarks, the loss
gradient norm-based metrics of individual training examples applied in image
classification fail to identify the important samples. To address this issue,
we propose a data pruning method by taking into consideration the training
dynamics on target regions using Dynamic Average Dice (DAD) score. To the best
of our knowledge, we are among the first to address the data importance in
dense labeling tasks in the field of medical image analysis, making the
following contributions: (1) investigating the underlying causes with rigorous
empirical analysis, and (2) determining effective data pruning approach in
dense labeling problems. Our solution can be used as a strong yet simple
baseline to select important examples for medical image segmentation with
combined data sources.

本文研究了稠密标记问题，提出了一种数据修剪方法，通过考虑目标区域上的训练动态使用动态平均 Dice（DAD）得分，确定了医学图像分割中选择重要示例的简单而强大的基准方法。

以数据为中心的饮食：针对医学图像分割的有效多中心数据集剪枝

Data-Centric Diet: Effective Multi-center Dataset Pruning for Medical  Image Segmentation

Retrieval augmentation enables large language models to take advantage of
external knowledge, for example on tasks like question answering and data
imputation. However, the performance of such retrieval-augmented models is
limited by the data quality of their underlying retrieval corpus. In this
paper, we propose an algorithm based on multilinear extension for evaluating
the data importance of retrieved data points. There are exponentially many
terms in the multilinear extension, and one key contribution of this paper is a
polynomial time algorithm that computes exactly, given a retrieval-augmented
model with an additive utility function and a validation set, the data
importance of data points in the retrieval corpus using the multilinear
extension of the model's utility function. We further proposed an even more
efficient ({\epsilon}, {\delta})-approximation algorithm. Our experimental
results illustrate that we can enhance the performance of large language models
by only pruning or reweighting the retrieval corpus, without requiring further
training. For some tasks, this even allows a small model (e.g., GPT-JT),
augmented with a search engine API, to outperform GPT-3.5 (without retrieval
augmentation). Moreover, we show that weights based on multilinear extension
can be computed efficiently in practice (e.g., in less than ten minutes for a
corpus with 100 million elements).

通过修剪或重新权重检索语料库，可以提高大型语言模型的性能，无需进行进一步训练。在某些任务中，甚至允许较小的模型（例如 GPT-JT）通过搜索引擎 API 进行扩展，胜过未使用检索增强的 GPT-3.5。此外，我们展示了在实践中可以高效地计算基于多线性扩展的权重（例如，对于包含 1 亿个元素的语料库，计算时间少于十分钟）

通过数据重要性学习改进检索增强的大型语言模型

Improving Retrieval-Augmented Large Language Models via Data Importance  Learning

We present techniques to characterize which data is important to a
recommender system and which is not. Important data is data that contributes
most to the accuracy of the recommendation algorithm, while less important data
contributes less to the accuracy or even decreases it. Characterizing the
importance of data has two potential direct benefits: (1) increased privacy and
(2) reduced data management costs, including storage. For privacy, we enable
increased recommendation accuracy for comparable privacy levels using existing
data obfuscation techniques. For storage, our results indicate that we can
achieve large reductions in recommendation data and yet maintain recommendation
accuracy.
Our main technique is called differential data analysis. The name is inspired
by other sorts of differential analysis, such as differential power analysis
and differential cryptanalysis, where insight comes through analysis of
slightly differing inputs. In differential data analysis we chunk the data and
compare results in the presence or absence of each chunk. We present results
applying differential data analysis to two datasets and three different kinds
of attributes. The first attribute is called user hardship. This is a novel
attribute, particularly relevant to location datasets, that indicates how
burdensome a data point was to achieve. The second and third attributes are
more standard: timestamp and user rating. For user rating, we confirm previous
work concerning the increased importance to the recommender of data
corresponding to high and low user ratings.

介绍了一种以差分数据分析为主要方法的技术来评估推荐系统中的数据重要性及其效果，可通过数据模糊技术提高推荐的准确性、减少数据存储等成本，并通过两个数据集和三种不同类型的属性阐述了该技术的应用。