Decentralized and incomplete data sources are prevalent in real-world
applications, posing a formidable challenge for causal inference. These sources
cannot be consolidated into a single entity owing to privacy constraints, and
the presence of missing values within them can potentially introduce bias to
the causal estimands. We introduce a new approach for federated causal
inference from incomplete data, enabling the estimation of causal effects from
multiple decentralized and incomplete data sources. Our approach disentangles
the loss function into multiple components, each corresponding to a specific
data source with missing values. Our approach accounts for the missing data
under the missing at random assumption, while also estimating higher-order
statistics of the causal estimands. Our method recovers the conditional
distribution of missing confounders given the observed confounders from the
decentralized data sources to identify causal effects. Our framework estimates
heterogeneous causal effects without the sharing of raw training data among
sources, which helps to mitigate privacy risks. The efficacy of our approach is
demonstrated through a collection of simulated and real-world instances,
illustrating its potential and practicality.

通过分散的不完整数据源进行联邦因果推断，估计多个分散的不完整数据源中的因果效应。该方法使用丢失数据的条件分布来识别因果关系，并估计缺失随机假设下的更高阶统计量，同时回答隐私风险。通过模拟和真实世界示例的集合验证了该方法的功效和实用性。

不完整观测数据的因果效应联邦学习

Federated Learning of Causal Effects from Incomplete Observational Data

Federated Learning (FL) is a novel machine learning approach that allows the
model trainer to access more data samples, by training the model across
multiple decentralized data sources, while data access constraints are in
place. Such trained models can achieve significantly higher performance beyond
what can be done when trained on a single data source. As part of FL's
promises, none of the training data is ever transmitted to any central
location, ensuring that sensitive data remains local and private. These
characteristics make FL perfectly suited for large-scale applications in
healthcare, where a variety of compliance constraints restrict how data may be
handled, processed, and stored. Despite the apparent benefits of federated
learning, the heterogeneity in the local data distributions pose significant
challenges, and such challenges are even more pronounced in the case of
multilingual data providers. In this paper we present a federated learning
system for training a large-scale multi-lingual model suitable for fine-tuning
on downstream tasks such as medical entity tagging. Our work represents one of
the first such production-scale systems, capable of training across multiple
highly heterogeneous data providers, and achieving levels of accuracy that
could not be otherwise achieved by using central training with public data.
Finally, we show that the global model performance can be further improved by a
training step performed locally.

该论文介绍了一个针对医学实体标注等下游任务的大规模多语言模型的联邦学习系统，它能够跨越高度异质化的数据提供者进行训练，并通过本地训练步骤进一步提高全局模型的性能。

面向医学转录分析的联合多语言模型

Federated Multilingual Models for Medical Transcript Analysis

Federated learning (FL) is a popular technique to train machine learning (ML)
models on decentralized data sources. In order to sustain long-term
participation of data owners, it is important to fairly appraise each data
source and compensate data owners for their contribution to the training
process. The Shapley value (SV) defines a unique payoff scheme that satisfies
many desiderata for a data value notion. It has been increasingly used for
valuing training data in centralized learning. However, computing the SV
requires exhaustively evaluating the model performance on every subset of data
sources, which incurs prohibitive communication cost in the federated setting.
Besides, the canonical SV ignores the order of data sources during training,
which conflicts with the sequential nature of FL. This paper proposes a variant
of the SV amenable to FL, which we call the federated Shapley value. The
federated SV preserves the desirable properties of the canonical SV while it
can be calculated without incurring extra communication cost and is also able
to capture the effect of participation order on data value. We conduct a
thorough empirical study of the federated SV on a range of tasks, including
noisy label detection, adversarial participant detection, and data
summarization on different benchmark datasets, and demonstrate that it can
reflect the real utility of data sources for FL and has the potential to
enhance system robustness, security, and efficiency. We also report and analyze
"failure cases" and hope to stimulate future research.

本文提出了一种适用于联邦学习的 Shapley 值变体，称为联邦 Shapley 值，它可以计算每个数据源的贡献，能够反映数据源的实际效用并有潜力提高系统的鲁棒性，安全性和效率。