The growing richness of large-scale datasets has been crucial in driving the
rapid advancement and wide adoption of machine learning technologies. The
massive collection and usage of data, however, pose an increasing risk for
people's private and sensitive information due to either inadvertent
mishandling or malicious exploitation. Besides legislative solutions, many
technical approaches have been proposed towards data privacy protection.
However, they bear various limitations such as leading to degraded data
availability and utility, or relying on heuristics and lacking solid
theoretical bases. To overcome these limitations, we propose a formal
information-theoretic definition for this utility-preserving privacy protection
problem, and design a data-driven learnable data transformation framework that
is capable of selectively suppressing sensitive attributes from target datasets
while preserving the other useful attributes, regardless of whether or not they
are known in advance or explicitly annotated for preservation. We provide
rigorous theoretical analyses on the operational bounds for our framework, and
carry out comprehensive experimental evaluations using datasets of a variety of
modalities, including facial images, voice audio clips, and human activity
motion sensor signals. Results demonstrate the effectiveness and
generalizability of our method under various configurations on a multitude of
tasks.

我们提出了一个形式化的信息论定义，设计了一个可学习的数据转换框架，能够选择性地抑制目标数据集中的敏感属性，同时保留其他有用属性，无论其是否事先已知或明确注解以便保留。通过对多种模态的数据集进行全面的实验评估，包括人脸图像、语音音频片段和人体活动传感器信号，结果显示我们的方法在多种配置和任务下都具有有效性和普适性。

MaSS: 多属性选择性压制从信息论的角度实现数据的保用变换

MaSS: Multi-attribute Selective Suppression for Utility-preserving Data  Transformation from an Information-theoretic Perspective

The rapid development of large language models has led to the widespread
adoption of Retrieval-Augmented Generation (RAG), which integrates external
knowledge to alleviate knowledge bottlenecks and mitigate hallucinations.
However, the existing RAG paradigm inevitably suffers from the impact of flawed
information introduced during the retrieval phrase, thereby diminishing the
reliability and correctness of the generated outcomes. In this paper, we
propose Credibility-aware Generation (CAG), a universally applicable framework
designed to mitigate the impact of flawed information in RAG. At its core, CAG
aims to equip models with the ability to discern and process information based
on its credibility. To this end, we propose an innovative data transformation
framework that generates data based on credibility, thereby effectively
endowing models with the capability of CAG. Furthermore, to accurately evaluate
the models' capabilities of CAG, we construct a comprehensive benchmark
covering three critical real-world scenarios. Experimental results demonstrate
that our model can effectively understand and utilize credibility for
generation, significantly outperform other models with retrieval augmentation,
and exhibit resilience against the disruption caused by noisy documents,
thereby maintaining robust performance. Moreover, our model supports customized
credibility, offering a wide range of potential applications.

本研究提出了一个通用的 CAG 框架，旨在减轻 RAG 模型中引入的错误信息对生成结果的影响。通过创新的数据转换框架生成基于可靠性的数据，使模型具备辨别和处理信息可靠性的能力。实验证明，该模型在生成中能够有效理解和利用可靠性，显著优于其他带有检索增强的模型，并对噪声文档引起的干扰表现出韧性，从而保持稳健的性能。此外，该模型支持定制的可靠性，有广泛的潜在应用。