We study evaluating a policy under best- and worst-case perturbations to a
Markov decision process (MDP), given transition observations from the original
MDP, whether under the same or different policy. This is an important problem
when there is the possibility of a shift between historical and future
environments, due to e.g. unmeasured confounding, distributional shift, or an
adversarial environment. We propose a perturbation model that can modify
transition kernel densities up to a given multiplicative factor or its
reciprocal, which extends the classic marginal sensitivity model (MSM) for
single time step decision making to infinite-horizon RL. We characterize the
sharp bounds on policy value under this model, that is, the tightest possible
bounds given by the transition observations from the original MDP, and we study
the estimation of these bounds from such transition observations. We develop an
estimator with several appealing guarantees: it is semiparametrically
efficient, and remains so even when certain necessary nuisance functions such
as worst-case Q-functions are estimated at slow nonparametric rates. It is also
asymptotically normal, enabling easy statistical inference using Wald
confidence intervals. In addition, when certain nuisances are estimated
inconsistently we still estimate a valid, albeit possibly not sharp bounds on
the policy value. We validate these properties in numeric simulations. The
combination of accounting for environment shifts from train to test
(robustness), being insensitive to nuisance-function estimation
(orthogonality), and accounting for having only finite samples to learn from
(inference) together leads to credible and reliable policy evaluation.

在环境变化、干扰函数估计不一致和有限样本学习的情况下，本研究旨在评估策略值，并提出了一种扰动模型，可以根据转移观测对传统 MDP 进行边界估计。

健壮马氏决策过程中高效锐利的离策略评估

Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision  Processes

Visual counterfactual explanations (VCEs) in image space are an important
tool to understand decisions of image classifiers as they show under which
changes of the image the decision of the classifier would change. Their
generation in image space is challenging and requires robust models due to the
problem of adversarial examples. Existing techniques to generate VCEs in image
space suffer from spurious changes in the background. Our novel perturbation
model for VCEs together with its efficient optimization via our novel
Auto-Frank-Wolfe scheme yields sparse VCEs which lead to subtle changes
specific for the target class. Moreover, we show that VCEs can be used to
detect undesired behavior of ImageNet classifiers due to spurious features in
the ImageNet dataset.

通过我们的新型扰动模型和自动 Frank-Wolfe 方案的高效优化，可以生成稀疏的视觉对抗解释，用以检测 ImageNet 分类器中的假特征。

图像空间中的稀疏视觉对抗解释

Sparse Visual Counterfactual Explanations in Image Space

Cook's [J. Roy. Statist. Soc. Ser. B 48 (1986) 133--169] local influence
approach based on normal curvature is an important diagnostic tool for
assessing local influence of minor perturbations to a statistical model.
However, no rigorous approach has been developed to address two fundamental
issues: the selection of an appropriate perturbation and the development of
influence measures for objective functions at a point with a nonzero first
derivative. The aim of this paper is to develop a differential--geometrical
framework of a perturbation model (called the perturbation manifold) and
utilize associated metric tensor and affine curvatures to resolve these issues.
We will show that the metric tensor of the perturbation manifold provides
important information about selecting an appropriate perturbation of a model.
Moreover, we will introduce new influence measures that are applicable to
objective functions at any point. Examples including linear regression models
and linear mixed models are examined to demonstrate the effectiveness of using
new influence measures for the identification of influential observations.

本研究提出了一种利用微分几何框架的扰动模型，通过度量张量和仿射曲率解决局部影响分析中的扰动选择和影响测度问题，并引入新的影响测度，以检测影响观测值，研究了线性回归模型和线性混合模型。