Algorithmic and data-driven decisions and recommendations are commonly used
in high-stakes decision-making settings such as criminal justice, medicine, and
public policy. We investigate whether it would have been possible to improve a
security assessment algorithm employed during the Vietnam War, using outcomes
measured immediately after its introduction in late 1969. This empirical
application raises several methodological challenges that frequently arise in
high-stakes algorithmic decision-making. First, before implementing a new
algorithm, it is essential to characterize and control the risk of yielding
worse outcomes than the existing algorithm. Second, the existing algorithm is
deterministic, and learning a new algorithm requires transparent extrapolation.
Third, the existing algorithm involves discrete decision tables that are common
but difficult to optimize over.
To address these challenges, we introduce the Average Conditional Risk
(ACRisk), which first quantifies the risk that a new algorithmic policy leads
to worse outcomes for subgroups of individual units and then averages this over
the distribution of subgroups. We also propose a Bayesian policy learning
framework that maximizes the posterior expected value while controlling the
posterior expected ACRisk. This framework separates the estimation of
heterogeneous treatment effects from policy optimization, enabling flexible
estimation of effects and optimization over complex policy classes. We
characterize the resulting chance-constrained optimization problem as a
constrained linear programming problem. Our analysis shows that compared to the
actual algorithm used during the Vietnam War, the learned algorithm assesses
most regions as more secure and emphasizes economic and political factors over
military factors.

在研究中，我们调查了是否有可能在越南战争期间，通过使用 1969 年末引入之后立即测量到的结果来改进当时使用的安全评估算法。我们还提出了一种贝叶斯策略学习框架以及平均条件风险（ACRisk）来解决与高风险算法决策制定相关的挑战。研究结果显示，相比于越南战争期间实际使用的算法，学习算法将大多数地区评估为更安全，并且强调经济和政治因素胜过军事因素。