The real life time series are usually nonstationary, bringing a difficult
question of model adaptation. Classical approaches like ARMA-ARCH assume
arbitrary type of dependence. To avoid such bias, we will focus on recently
proposed agnostic philosophy of moving estimator: in time $t$ finding
parameters optimizing e.g. $F_t=\sum_{\tau<t} (1-\eta)^{t-\tau} \ln(\rho_\theta
(x_\tau))$ moving log-likelihood, evolving in time. It allows for example to
estimate parameters using inexpensive exponential moving averages (EMA), like
absolute central moments $E[|x-\mu|^p]$ evolving for one or multiple powers
$p\in\mathbb{R}^+$ using $m_{p,t+1} = m_{p,t} + \eta (|x_t-\mu_t|^p-m_{p,t})$.
Application of such general adaptive methods of moments will be presented on
Student's t-distribution, popular especially in economical applications, here
applied to log-returns of DJIA companies. While standard ARMA-ARCH approaches
provide evolution of $\mu$ and $\sigma$, here we also get evolution of $\nu$
describing $\rho(x)\sim |x|^{-\nu-1}$ tail shape, probability of extreme events
- which might turn out catastrophic, destabilizing the market.

该研究讨论了如何使用移动估计器来适应非平稳的实际时间序列数据，并以学生 t - 分布为例进行演示，以获得在不同时间单位内的泰尔指数演变， 以概括市场的稳定性。

非平稳时间序列的矩方法移动估计自适应学生 t 分布

Adaptive Student's t-distribution with method of moments moving  estimator for nonstationary time series

Identifying causality is a challenging task in many data-intensive scenarios.
Many algorithms have been proposed for this critical task. However, most of
them consider the learning algorithms for directed acyclic graph (DAG) of
Bayesian network (BN). These BN-based models only have limited causal
explainability because of the issue of Markov equivalence class. Moreover, they
are dependent on the assumption of stationarity, whereas many sampling time
series from complex system are nonstationary. The nonstationary time series
bring dataset shift problem, which leads to the unsatisfactory performances of
these algorithms. To fill these gaps, a novel causation model named Unique
Causal Network (UCN) is proposed in this paper. Different from the previous
BN-based models, UCN considers the influence of time delay, and proves the
uniqueness of obtained network structure, which addresses the issue of Markov
equivalence class. Furthermore, based on the decomposability property of UCN, a
higher-order causal entropy (HCE) algorithm is designed to identify the
structure of UCN in a distributed way. HCE algorithm measures the strength of
causality by using nearest-neighbors entropy estimator, which works well on
nonstationary time series. Finally, lots of experiments validate that HCE
algorithm achieves state-of-the-art accuracy when time series are
nonstationary, compared to the other baseline algorithms.

本文提出了一种新的因果模型 Unique Causal Network，解决了基于贝叶斯网络的模型在因果解释性上存在的问题，同时提出了基于高阶因果熵算法的分布式 UCN 结构识别方法，用于处理非平稳时间序列数据，实验证明该方法在非平稳时间序列数据上具有最先进的准确性。

从非静态时间序列中识别独特因果网络

Identifying Unique Causal Network from Nonstationary Time Series

We consider reinforcement learning (RL) in episodic Markov decision processes
(MDPs) with linear function approximation under drifting environment.
Specifically, both the reward and state transition functions can evolve over
time, as long as their respective total variations, quantified by suitable
metrics, do not exceed certain \textit{variation budgets}. We first develop the
$\texttt{LSVI-UCB-Restart}$ algorithm, an optimistic modification of
least-squares value iteration combined with periodic restart, and establish its
dynamic regret bound when variation budgets are known. We then propose a
parameter-free algorithm, $\texttt{Ada-LSVI-UCB-Restart}$, that works without
knowing the variation budgets, but with a slightly worse dynamic regret bound.
We also derive the first minimax dynamic regret lower bound for nonstationary
MDPs to show that our proposed algorithms are near-optimal. As a byproduct, we
establish a minimax regret lower bound for linear MDPs, which is unsolved by
\cite{jin2020provably}. In addition, we provide numerical experiments to
demonstrate the effectiveness of our proposed algorithms. As far as we know,
this is the first dynamic regret analysis in nonstationary reinforcement
learning with function approximation.

这篇研究采用线性函数逼近的方法来应用强化学习在马尔科夫决策过程中，通过衡量合适的指标来保证奖励和状态转移函数变化的幅度不超过一定的上限，提出了两种最优算法：LSVI-UCB-Restart 和 Ada-LSVI-UCB-Restart。该研究还为非平稳 MDP 和线性 MDP 提供了动态遗憾分析的理论支持，并进行了有效性验证。