Early warning systems (EWS) are prediction algorithms that have recently
taken a central role in efforts to improve graduation rates in public schools
across the US. These systems assist in targeting interventions at individual
students by predicting which students are at risk of dropping out. Despite
significant investments and adoption, there remain significant gaps in our
understanding of the efficacy of EWS. In this work, we draw on nearly a
decade's worth of data from a system used throughout Wisconsin to provide the
first large-scale evaluation of the long-term impact of EWS on graduation
outcomes.
We present evidence that risk assessments made by the prediction system are
highly accurate, including for students from marginalized backgrounds. Despite
the system's accuracy and widespread use, we find no evidence that it has led
to improved graduation rates. We surface a robust statistical pattern that can
explain why these seemingly contradictory insights hold. Namely, environmental
features, measured at the level of schools, contain significant signal about
dropout risk. Within each school, however, academic outcomes are essentially
independent of individual student performance. This empirical observation
indicates that assigning all students within the same school the same
probability of graduation is a nearly optimal prediction.
Our work provides an empirical backbone for the robust, qualitative
understanding among education researchers and policy-makers that dropout is
structurally determined. The primary barrier to improving outcomes lies not in
identifying students at risk of dropping out within specific schools, but
rather in overcoming structural differences across different school districts.
Our findings indicate that we should carefully evaluate the decision to fund
early warning systems without also devoting resources to interventions tackling
structural barriers.

使用威斯康辛州近十年的数据进行大规模评估后发现，尽管风险评估系统的准确性很高，但没有证据表明其提高了毕业率。结构约束是改善教育成果的主要障碍。

威斯康辛公立学校的社会预测困难教训

Difficult Lessons on Social Prediction from Wisconsin Public Schools

India accounts for 11% of maternal deaths globally where a woman dies in
childbirth every fifteen minutes. Lack of access to preventive care information
is a significant problem contributing to high maternal morbidity and mortality
numbers, especially in low-income households. We work with ARMMAN, a non-profit
based in India, to further the use of call-based information programs by
early-on identifying women who might not engage on these programs that are
proven to affect health parameters positively.We analyzed anonymized
call-records of over 300,000 women registered in an awareness program created
by ARMMAN that uses cellphone calls to regularly disseminate health related
information. We built robust deep learning based models to predict short term
and long term dropout risk from call logs and beneficiaries' demographic
information. Our model performs 13% better than competitive baselines for
short-term forecasting and 7% better for long term forecasting. We also discuss
the applicability of this method in the real world through a pilot validation
that uses our method to perform targeted interventions.

本研究使用深度学习模型从 ARMMAN 基于电话的信息项目中预测短期和长期的产妇退出率，以解决印度母亲死亡率过高的问题。模型表现比竞争基线模型分别提高了 13％和 7％。此方法在现实应用中有潜力进行有针对性的干预措施。