We consider evaluating and training a new policy for the evaluation data by
using the historical data obtained from a different policy. The goal of
off-policy evaluation (OPE) is to estimate the expected reward of a new policy
over the evaluation data, and that of off-policy learning (OPL) is to find a
new policy that maximizes the expected reward over the evaluation data.
Although the standard OPE and OPL assume the same distribution of covariate
between the historical and evaluation data, a covariate shift often exists,
i.e., the distribution of the covariate of the historical data is different
from that of the evaluation data. In this paper, we derive the efficiency bound
of OPE under a covariate shift. Then, we propose doubly robust and efficient
estimators for OPE and OPL under a covariate shift by using a nonparametric
estimator of the density ratio between the historical and evaluation data
distributions. We also discuss other possible estimators and compare their
theoretical properties. Finally, we confirm the effectiveness of the proposed
estimators through experiments.

该研究旨在探讨如何利用历史数据评估和训练新政策，避免协变量偏移的问题，并提出有效的估算器和比较其理论性质。

基于协变量偏移的对外有效性的非策略评估与学习

Off-Policy Evaluation and Learning for External Validity under a  Covariate Shift

Hawkes (1971) introduced a powerful multivariate point process model of
mutually exciting processes to explain causal structure in data. In this paper
it is shown that the Granger causality structure of such processes is fully
encoded in the corresponding link functions of the model. A new nonparametric
estimator of the link functions based on a time-discretized version of the
point process is introduced by using an infinite order autoregression.
Consistency of the new estimator is derived. The estimator is applied to
simulated data and to neural spike train data from the spinal dorsal horn of a
rat.

该论文介绍了一种基于 Hawkes 模型的多变量点过程，展示了该过程的 Granger 因果结构完全被该模型的相应链接函数所编码；并提出了一种基于无限阶自回归的时间离散版本的点过程的链接函数的新的非参数估计器，应用于模拟数据和大鼠脊髓背角的神经脉冲数据，并验证了该估计器的一致性。

多元 Hawkes 过程的图形建模与非参数链接函数

Graphical Modeling for Multivariate Hawkes Processes with Nonparametric  Link Functions

We demonstrate that a popular class of nonparametric mutual information (MI)
estimators based on k-nearest-neighbor graphs requires number of samples that
scales exponentially with the true MI. Consequently, accurate estimation of MI
between two strongly dependent variables is possible only for prohibitively
large sample size. This important yet overlooked shortcoming of the existing
estimators is due to their implicit reliance on local uniformity of the
underlying joint distribution. We introduce a new estimator that is robust to
local non-uniformity, works well with limited data, and is able to capture
relationship strengths over many orders of magnitude. We demonstrate the
superior performance of the proposed estimator on both synthetic and real-world
data.

提出了一种新的相互信息非参数估计值，解决了其他方法基于局部分布均匀性计算相互信息的局限性，对于很少的数据也可以准确地估计两个相关变量之间的相互信息，并且在合成和真实世界数据上表现出卓越的性能。

高度相关变量互信息的有效估计

Efficient Estimation of Mutual Information for Strongly Dependent  Variables

Most machine learning algorithms, such as classification or regression, treat
the individual data point as the object of interest. Here we consider extending
machine learning algorithms to operate on groups of data points. We suggest
treating a group of data points as an i.i.d. sample set from an underlying
feature distribution for that group. Our approach employs kernel machines with
a kernel on i.i.d. sample sets of vectors. We define certain kernel functions
on pairs of distributions, and then use a nonparametric estimator to
consistently estimate those functions based on sample sets. The projection of
the estimated Gram matrix to the cone of symmetric positive semi-definite
matrices enables us to use kernel machines for classification, regression,
anomaly detection, and low-dimensional embedding in the space of distributions.
We present several numerical experiments both on real and simulated datasets to
demonstrate the advantages of our new approach.

该论文提出了一种基于核函数的机器学习算法，可以通过对数据集的分组进行处理，采用独立同分布的样本集作为数据点，利用非参数估计器提取核函数特征从而实现多种分类、回归和异常检测等任务。