Neural ordinary differential equations (ODEs) are widely recognized as the
standard for modeling physical mechanisms, which help to perform approximate
inference in unknown physical or biological environments. In partially
observable (PO) environments, how to infer unseen information from raw
observations puzzled the agents. By using a recurrent policy with a compact
context, context-based reinforcement learning provides a flexible way to
extract unobservable information from historical transitions. To help the agent
extract more dynamics-related information, we present a novel ODE-based
recurrent model combines with model-free reinforcement learning (RL) framework
to solve partially observable Markov decision processes (POMDPs). We
experimentally demonstrate the efficacy of our methods across various PO
continuous control and meta-RL tasks. Furthermore, our experiments illustrate
that our method is robust against irregular observations, owing to the ability
of ODEs to model irregularly-sampled time series.

通过结合神经常微分方程和无模型强化学习，我们提出了一种新颖的基于 ODE 的循环模型用于解决部分可观察的马尔可夫决策过程，通过模型推断从历史过渡中提取不可观测的动态相关信息，并通过多个实验验证了方法的有效性和鲁棒性，尤其在处理不规则采样的时间序列方面。

基于 ODE 的无模型循环强化学习在 POMDP 中的应用

ODE-based Recurrent Model-free Reinforcement Learning for POMDPs

Modelling irregularly-sampled time series (ISTS) is challenging because of
missing values. Most existing methods focus on handling ISTS by converting
irregularly sampled data into regularly sampled data via imputation. These
models assume an underlying missing mechanism leading to unwanted bias and
sub-optimal performance. We present SLAN (Switch LSTM Aggregate Network), which
utilizes a pack of LSTMs to model ISTS without imputation, eliminating the
assumption of any underlying process. It dynamically adapts its architecture on
the fly based on the measured sensors. SLAN exploits the irregularity
information to capture each sensor's local summary explicitly and maintains a
global summary state throughout the observational period. We demonstrate the
efficacy of SLAN on publicly available datasets, namely, MIMIC-III, Physionet
2012 and Physionet 2019. The code is available at
this https URL

使用 SLAN（Switch LSTM Aggregate Network）模型，不需要填充缺失值即可对不规则采样时间序列（ISTS）进行建模，通过动态调整模型架构、利用不规则信息和全局摘要状态来实现对每个传感器的建模。

无需插补的不规则采样时间序列建模

Modelling Irregularly Sampled Time Series Without Imputation

Irregularly-sampled time series occur in many domains including healthcare.
They can be challenging to model because they do not naturally yield a
fixed-dimensional representation as required by many standard machine learning
models. In this paper, we consider irregular sampling from the perspective of
missing data. We model observed irregularly-sampled time series data as a
sequence of index-value pairs sampled from a continuous but unobserved
function. We introduce an encoder-decoder framework for learning from such
generic indexed sequences. We propose learning methods for this framework based
on variational autoencoders and generative adversarial networks. For continuous
irregularly-sampled time series, we introduce continuous convolutional layers
that can efficiently interface with existing neural network architectures.
Experiments show that our models are able to achieve competitive or better
classification results on irregularly-sampled multivariate time series compared
to recent RNN models while offering significantly faster training times.

本文介绍一种基于变分自编码器和生成对抗网络的编码器 - 解码器框架，用于学习不规则采样的时间序列数据，提出了一种连续卷积层用于与现有神经网络体系结构高效对接。实验证明，该模型在不规则多元时间序列分类的结果上可以比最近的 RNN 模型表现得更好，同时具有显著更快的训练速度。