Free-text rationales play a pivotal role in explainable NLP, bridging the
knowledge and reasoning gaps behind a model's decision-making. However, due to
the diversity of potential reasoning paths and a corresponding lack of
definitive ground truth, their evaluation remains a challenge. Existing
evaluation metrics rely on the degree to which a rationale supports a target
label, but we find these fall short in evaluating rationales that inadvertently
leak the labels. To address this problem, we propose RORA, a Robust free-text
Rationale evaluation against label leakage. RORA quantifies the new information
supplied by a rationale to justify the label. This is achieved by assessing the
conditional V-information \citep{hewitt-etal-2021-conditional} with a
predictive family robust against leaky features that can be exploited by a
small model. RORA consistently outperforms existing approaches in evaluating
human-written, synthetic, or model-generated rationales, particularly
demonstrating robustness against label leakage. We also show that RORA aligns
well with human judgment, providing a more reliable and accurate measurement
across diverse free-text rationales.

通过评估条件性 V - 信息以及对容易被小型模型利用的敏感特征进行鲁棒性评估，我们提出了一种针对标签泄漏的强韧的自由文本基于理由的评估方法 RORA，它在评估人类编写、合成或模型生成的基于理由时提供了更可靠和准确的度量，尤其是对于标签泄漏展现出了鲁棒性。

RORA: 强健自由文本理由评估

RORA: Robust Free-Text Rationale Evaluation

Vertical federated learning (VFL) enables multiple parties with disjoint
features of a common user set to train a machine learning model without sharing
their private data. Tree-based models have become prevalent in VFL due to their
interpretability and efficiency. However, the vulnerability of tree-based VFL
has not been sufficiently investigated. In this study, we first introduce a
novel label inference attack, ID2Graph, which utilizes the sets of record-IDs
assigned to each node (i.e., instance space) to deduce private training labels.
The ID2Graph attack generates a graph structure from training samples, extracts
communities from the graph, and clusters the local dataset using community
information. To counteract label leakage from the instance space, we propose an
effective defense mechanism, ID-LMID, which prevents label leakage by focusing
on mutual information regularization. Comprehensive experiments conducted on
various datasets reveal that the ID2Graph attack presents significant risks to
tree-based models such as Random Forest and XGBoost. Further evaluations on
these benchmarks demonstrate that ID-LMID effectively mitigates label leakage
in such instances.

垂直联邦学习中存在的以树模型为基础的标签推断攻击 (ID2Graph) 对 Random Forest 和 XGBoost 等树模型存在显著风险，而 ID-LMID 机制能够有效降低标签泄露。

消除基于树的垂直联邦学习中的标签泄漏

Eliminating Label Leakage in Tree-Based Vertical Federated Learning

Two-party split learning is a popular technique for learning a model across
feature-partitioned data. In this work, we explore whether it is possible for
one party to steal the private label information from the other party during
split training, and whether there are methods that can protect against such
attacks. Specifically, we first formulate a realistic threat model and propose
a privacy loss metric to quantify label leakage in split learning. We then show
that there exist two simple yet effective methods within the threat model that
can allow one party to accurately recover private ground-truth labels owned by
the other party. To combat these attacks, we propose several random
perturbation techniques, including $\texttt{Marvell}$, an approach that
strategically finds the structure of the noise perturbation by minimizing the
amount of label leakage (measured through our quantification metric) of a
worst-case adversary. We empirically demonstrate the effectiveness of our
protection techniques against the identified attacks, and show that
$\texttt{Marvell}$ in particular has improved privacy-utility tradeoffs
relative to baseline approaches.

本研究探索了在双方分别拥有数据的场景下，一方能否窃取另一方的标签信息，并提出了量化泄露度的隐私损失度量、防御方法和基于随机扰动的修复技术 $	exttt {Marvell}$。实验证明，该技术具有更好的隐私效果和使用效用权衡。