In this study, we introduce a novel two-pronged scoring system to measure
individual and systemic bias in immigration courts under the U.S. Executive
Office of Immigration Review (EOIR). We analyze nearly 6 million immigration
court proceedings and 228 case features to build on prior research showing that
U.S. asylum decisions vary dramatically based on factors that are extraneous to
the merits of a case. We close a critical gap in the literature of variability
metrics that can span space and time. Using predictive modeling, we explain
58.54% of the total decision variability using two metrics: partisanship and
inter-judge cohort consistency. Thus, whether the EOIR grants asylum to an
applicant or not depends in majority on the combined effects of the political
climate and the individual variability of the presiding judge - not the
individual merits of the case. Using time series analysis, we also demonstrate
that partisanship increased in the early 1990s but plateaued following the turn
of the century. These conclusions are striking to the extent that they diverge
from the U.S. immigration system's commitments to independence and due process.
Our contributions expose systemic inequities in the U.S. asylum decision-making
process, and we recommend improved and standardized variability metrics to
better diagnose and monitor these issues.

我们引入了一种新的，分为两个层面的评分系统对美国移民法庭下属的执行办公室进行个体和系统的偏见度量。通过分析将近 600 万个移民法庭诉讼和 228 种案例特征，我们展示了美国避难决策取决于政治气候和审判长个体变异性的结合效应，而不是案件本身的个体优点，并暴露了美国避难决策制定过程中的制度性不公平和建议改进和标准化可变性指标以更好地诊断和监测这些问题。

美国庇护案件中的偏见、一致性和党派性：机器学习分析移民法庭决定中的外部因素

Bias, Consistency, and Partisanship in U.S. Asylum Cases: A Machine  Learning Analysis of Extraneous Factors in Immigration Court Decisions

We first adopt a human-guided machine learning framework to develop a new
dataset for hyperpartisan news title detection with 2,200 manually labeled and
1.8 million machine-labeled titles that were posted from 2014 to the present by
nine representative media organizations across three media bias groups - Left,
Central, and Right in an active learning manner. The fine-tuned
transformer-based language model achieves an overall accuracy of 0.84 and an F1
score of 0.78 on an external validation set. Next, we conduct a computational
analysis to quantify the extent and dynamics of partisanship in news titles.
While some aspects are as expected, our study reveals new or nuanced
differences between the three media groups. We find that overall the Right
media tends to use proportionally more hyperpartisan titles. Roughly around the
2016 Presidential Election, the proportions of hyperpartisan titles increased
in all media bias groups where the relative increase in the proportion of
hyperpartisan titles of the Left media was the most. We identify three major
topics including foreign issues, political systems, and societal issues that
are suggestive of hyperpartisanship in news titles using logistic regression
models and the Shapley values. Through an analysis of the topic distribution,
we find that societal issues gradually receive more attention from all media
groups. We further apply a lexicon-based language analysis tool to the titles
of each topic and quantify the linguistic distance between any pairs of the
three media groups. Three distinct patterns are discovered. The Left media is
linguistically more different from Central and Right in terms of foreign
issues. The linguistic distance between the three media groups becomes smaller
over recent years. In addition, a seasonal pattern where linguistic difference
is associated with elections is observed for societal issues.

本文采取人工辅助机器学习框架，针对具有代表性的 3 个主流媒体倾向组（左，中，右）在 2014 年至今发布的 2.2 万份手动标记和 180 万份机器标记的新闻标题进行了超党派新闻标题检测的数据集开发，并在外部验证集上取得了 0.84 的准确度和 0.78 的 F1 得分；针对新闻标题中的党派性进行了计算分析，发现右翼媒体倾向于使用比例更高的超党派标题；通过逻辑回归模型和 Shapley 值确定了几个主题，并应用基于词库的语言分析工具对每个主题的标题进行了分析，发现左派、中派和右派媒体在外交、政治和社会问题方面的语言差异逐渐减小。