This paper studies reinforcement learning (RL) in doubly inhomogeneous
environments under temporal non-stationarity and subject heterogeneity. In a
number of applications, it is commonplace to encounter datasets generated by
system dynamics that may change over time and population, challenging
high-quality sequential decision making. Nonetheless, most existing RL
solutions require either temporal stationarity or subject homogeneity, which
would result in sub-optimal policies if both assumptions were violated. To
address both challenges simultaneously, we propose an original algorithm to
determine the ``best data chunks" that display similar dynamics over time and
across individuals for policy learning, which alternates between most recent
change point detection and cluster identification. Our method is general, and
works with a wide range of clustering and change point detection algorithms. It
is multiply robust in the sense that it takes multiple initial estimators as
input and only requires one of them to be consistent. Moreover, by borrowing
information over time and population, it allows us to detect weaker signals and
has better convergence properties when compared to applying the clustering
algorithm per time or the change point detection algorithm per subject.
Empirically, we demonstrate the usefulness of our method through extensive
simulations and a real data application.

本文研究了强化学习在双重非均质环境下的应用，提出了一种用于确定数据块并识别聚类的算法，以达到高质量序列决策的目的。

双重非齐次强化学习

Doubly Inhomogeneous Reinforcement Learning

The task of temporally detecting and segmenting actions in untrimmed videos
has seen an increased attention recently. One problem in this context arises
from the need to define and label action boundaries to create annotations for
training which is very time and cost intensive. To address this issue, we
propose an unsupervised approach for learning action classes from untrimmed
video sequences. To this end, we use a continuous temporal embedding of
framewise features to benefit from the sequential nature of activities. Based
on the latent space created by the embedding, we identify clusters of temporal
segments across all videos that correspond to semantic meaningful action
classes. The approach is evaluated on three challenging datasets, namely the
Breakfast dataset, YouTube Instructions, and the 50Salads dataset. While
previous works assumed that the videos contain the same high level activity, we
furthermore show that the proposed approach can also be applied to a more
general setting where the content of the videos is unknown.

该研究提出了一种基于连续时间嵌入的无监督学习方法，通过鉴别视觉序列中课程的聚类段以实现发现非结构化视频中的动作。该方法被评估在三个数据集上，可以适用于未知情景下的视觉内容分析

连续时间嵌入的无监督动作类别学习

Unsupervised learning of action classes with continuous temporal  embedding

Detecting clusters or communities in large real-world graphs such as large
social or information networks is a problem of considerable interest. In
practice, one typically chooses an objective function that captures the
intuition of a network cluster as set of nodes with better internal
connectivity than external connectivity, and then one applies approximation
algorithms or heuristics to extract sets of nodes that are related to the
objective function and that "look like" good communities for the application of
interest. In this paper, we explore a range of network community detection
methods in order to compare them and to understand their relative performance
and the systematic biases in the clusters they identify. We evaluate several
common objective functions that are used to formalize the notion of a network
community, and we examine several different classes of approximation algorithms
that aim to optimize such objective functions. In addition, rather than simply
fixing an objective and asking for an approximation to the best cluster of any
size, we consider a size-resolved version of the optimization problem.
Considering community quality as a function of its size provides a much finer
lens with which to examine community detection algorithms, since objective
functions and approximation algorithms often have non-obvious size-dependent
behavior.

本文探讨了一些网络社区检测方法，比较它们的性能和系统偏差；评估了用于形式化网络社区概念的几种常见目标函数，并研究了几种旨在优化这些目标函数的近似算法。此外，本论文还考虑了问题的大小解决版本，从社区大小的角度来考虑社区质量，以更好地检验社区检测算法，因为目标函数和近似算法通常具有不明显的大小依赖行为。