We study evaluating a policy under best- and worst-case perturbations to a
Markov decision process (MDP), given transition observations from the original
MDP, whether under the same or different policy. This is an important problem
when there is the possibility of a shift between historical and future
environments, due to e.g. unmeasured confounding, distributional shift, or an
adversarial environment. We propose a perturbation model that can modify
transition kernel densities up to a given multiplicative factor or its
reciprocal, which extends the classic marginal sensitivity model (MSM) for
single time step decision making to infinite-horizon RL. We characterize the
sharp bounds on policy value under this model, that is, the tightest possible
bounds given by the transition observations from the original MDP, and we study
the estimation of these bounds from such transition observations. We develop an
estimator with several appealing guarantees: it is semiparametrically
efficient, and remains so even when certain necessary nuisance functions such
as worst-case Q-functions are estimated at slow nonparametric rates. It is also
asymptotically normal, enabling easy statistical inference using Wald
confidence intervals. In addition, when certain nuisances are estimated
inconsistently we still estimate a valid, albeit possibly not sharp bounds on
the policy value. We validate these properties in numeric simulations. The
combination of accounting for environment shifts from train to test
(robustness), being insensitive to nuisance-function estimation
(orthogonality), and accounting for having only finite samples to learn from
(inference) together leads to credible and reliable policy evaluation.

在环境变化、干扰函数估计不一致和有限样本学习的情况下，本研究旨在评估策略值，并提出了一种扰动模型，可以根据转移观测对传统 MDP 进行边界估计。

健壮马氏决策过程中高效锐利的离策略评估

Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision  Processes

In this paper we explore the concept of sequential inductive prediction
intervals using theory from sequential testing. We furthermore introduce a
3-parameter PAC definition of prediction intervals that allows us via
simulation to achieve almost sharp bounds with high probability.

通过使用来自顺序测试理论的观点，本文探索了顺序归纳预测区间的概念，并引入了一个 3 参数 PAC 定义的预测区间，通过模拟实现几乎确定性的边界。

顺序归纳预测区间

Sequential inductive prediction intervals

This paper deals with the problem of estimating the probabilities of
causation when treatment and effect are not binary. Tian and Pearl derived
sharp bounds for the probability of necessity and sufficiency (PNS), the
probability of sufficiency (PS), and the probability of necessity (PN) using
experimental and observational data. In this paper, we provide theoretical
bounds for all types of probabilities of causation to multivalued treatments
and effects. We further discuss examples where our bounds guide practical
decisions and use simulation studies to evaluate how informative the bounds are
for various combinations of data.

本文提供了理论边界，以估算多价治疗和效应的因果概率，并使用模拟研究评估了这些边界对各种数据组合的信息性，以及对实际决策的指导作用。