Social media may disseminate medical claims that highlight misleading
correlations between social identifiers and diseases due to not accounting for
structural determinants of health. Our research aims to identify biased medical
claims on Twitter and measure their spread. We propose a machine learning
framework that uses two models in tandem: RoBERTa to detect medical claims and
DistilBERT to classify bias. After identifying original biased medical claims,
we conducted a retweet cascade analysis, computing their individual reach and
rate of spread. Tweets containing biased claims were found to circulate faster
and further than unbiased claims.

社交媒体上可能会传播强调社会识别因素与疾病之间误导性相关性的医疗声明，而不考虑健康的结构性决定因素。我们的研究旨在识别 Twitter 上的偏见医疗声明并测量它们的传播程度。我们提出了一个机器学习框架，使用 RoBERTa 模型检测医疗声明，使用 DistilBERT 模型分类偏见。在识别出原始的有偏见医疗声明后，我们进行了转推级联分析，计算它们的个体传播范围和传播速率。我们发现，包含偏见声明的推文传播得比无偏见声明的更快、更远。

基于推特的偏见医疗声称级联检测

Towards Detecting Cascades of Biased Medical Claims on Twitter

Sentiment Analysis Systems (SASs) are data-driven Artificial Intelligence
(AI) systems that, given a piece of text, assign one or more numbers conveying
the polarity and emotional intensity expressed in the input. Like other
automatic machine learning systems, they have also been known to exhibit model
uncertainty where a (small) change in the input leads to drastic swings in the
output. This can be especially problematic when inputs are related to protected
features like gender or race since such behavior can be perceived as a lack of
fairness, i.e., bias. We introduce a novel method to assess and rate SASs where
inputs are perturbed in a controlled causal setting to test if the output
sentiment is sensitive to protected variables even when other components of the
textual input, e.g., chosen emotion words, are fixed. We then use the result to
assign labels (ratings) at fine-grained and overall levels to convey the
robustness of the SAS to input changes. The ratings serve as a principled basis
to compare SASs and choose among them based on behavior. It benefits all users,
especially developers who reuse off-the-shelf SASs to build larger AI systems
but do not have access to their code or training data to compare.

本文介绍了一种评估情感分析系统漏洞的新方法，通过在受保护变量上控制调节输入以测试输出的情感是否对其敏感，给 SASs 分配标签（评分），以此评估其对输入变化的强度和鲁棒性，提高公平性，尤其对于开发人员寻找有效的信息来构建更大型的 AI 系统非常有用。

利用因果关系分析评分情感分析系统的偏差

Rating Sentiment Analysis Systems for Bias through a Causal Lens

When a model's performance differs across socially or culturally relevant
groups--like race, gender, or the intersections of many such groups--it is
often called "biased." While much of the work in algorithmic fairness over the
last several years has focused on developing various definitions of model
fairness (the absence of group-wise model performance disparities) and
eliminating such "bias," much less work has gone into rigorously measuring it.
In practice, it important to have high quality, human digestible measures of
model performance disparities and associated uncertainty quantification about
them that can serve as inputs into multi-faceted decision-making processes. In
this paper, we show both mathematically and through simulation that many of the
metrics used to measure group-wise model performance disparities are themselves
statistically biased estimators of the underlying quantities they purport to
represent. We argue that this can cause misleading conclusions about the
relative group-wise model performance disparities along different dimensions,
especially in cases where some sensitive variables consist of categories with
few members. We propose the "double-corrected" variance estimator, which
provides unbiased estimates and uncertainty quantification of the variance of
model performance across groups. It is conceptually simple and easily
implementable without statistical software package or numerical optimization.
We demonstrate the utility of this approach through simulation and show on a
real dataset that while statistically biased estimators of group-wise model
performance disparities indicate statistically significant differences, when
accounting for statistical bias in the estimator, the estimated between-group
disparities are no longer statistically significant.

通过数学和模拟，作者表明现有的用于测量群体间的模型性能差异的度量大多是基于统计偏差的估计量，因此提出了一种 “双校正” 方差估计器，提供了模型性能在群体间变化的无偏估计和不确定性量化，这种方法在现实数据集上的应用表明，在统计偏差的影响下，群组间的性能差异不再显著。