A promising characteristic of Deep Reinforcement Learning (DRL) is its
capability to learn optimal policy in an end-to-end manner without relying on
feature engineering. However, most approaches assume a fully observable state
space, i.e. fully observable Markov Decision Processes (MDPs). In real-world
robotics, this assumption is unpractical, because of issues such as sensor
sensitivity limitations and sensor noise, and the lack of knowledge about
whether the observation design is complete or not. These scenarios lead to
Partially Observable MDPs (POMDPs). In this paper, we propose
Long-Short-Term-Memory-based Twin Delayed Deep Deterministic Policy Gradient
(LSTM-TD3) by introducing a memory component to TD3, and compare its
performance with other DRL algorithms in both MDPs and POMDPs. Our results
demonstrate the significant advantages of the memory component in addressing
POMDPs, including the ability to handle missing and noisy observation data.

本文介绍了一种基于 LSTM-TD3 的方法，该方法引入了记忆组件以应对部分可观察 MDPs，相比其他 DRL 算法，在具有部分可观察 MDPs 的情况下，该方法具有显著的优势，包括处理丢失和噪声观察数据的能力。

基于记忆的深度强化学习在 POMDPs 中的应用

Memory-based Deep Reinforcement Learning for POMDPs

Multi-agent motion prediction is challenging because it aims to foresee the
future trajectories of multiple agents (\textit{e.g.} pedestrians)
simultaneously in a complicated scene. Existing work addressed this challenge
by either learning social spatial interactions represented by the positions of
a group of pedestrians, while ignoring their temporal coherence (\textit{i.e.}
dependencies between different long trajectories), or by understanding the
complicated scene layout (\textit{e.g.} scene segmentation) to ensure safe
navigation. However, unlike previous work that isolated the spatial
interaction, temporal coherence, and scene layout, this paper designs a new
mechanism, \textit{i.e.}, Dynamic and Static Context-aware Motion Predictor
(DSCMP), to integrates these rich information into the long-short-term-memory
(LSTM). It has three appealing benefits. (1) DSCMP models the dynamic
interactions between agents by learning both their spatial positions and
temporal coherence, as well as understanding the contextual scene layout.(2)
Different from previous LSTM models that predict motions by propagating hidden
features frame by frame, limiting the capacity to learn correlations between
long trajectories, we carefully design a differentiable queue mechanism in
DSCMP, which is able to explicitly memorize and learn the correlations between
long trajectories. (3) DSCMP captures the context of scene by inferring latent
variable, which enables multimodal predictions with meaningful semantic scene
layout. Extensive experiments show that DSCMP outperforms state-of-the-art
methods by large margins, such as 9.05\% and 7.62\% relative improvements on
the ETH-UCY and SDD datasets respectively.

使用动态和静态上下文感知运动预测器 (DSCMP) 将富含信息的空间交互作用、时间相依性、场景布局集成到长项短记忆（LSTM）中，学习多个代理 (如行人) 的未来轨迹，具有动态交互、学习 long-trajectory 关联性和推断场景潜在变量等优点。相比现有方法，DSCMP 在 ETH-UCY 和 SDD 数据集上相对改进分别为 9.05% 和 7.62%。

多智能体运动预测的动态和静态上下文感知 LSTM

Dynamic and Static Context-aware LSTM for Multi-agent Motion Prediction

In this paper we address the problem of human action recognition from video
sequences. Inspired by the exemplary results obtained via automatic feature
learning and deep learning approaches in computer vision, we focus our
attention towards learning salient spatial features via a convolutional neural
network (CNN) and then map their temporal relationship with the aid of
Long-Short-Term-Memory (LSTM) networks. Our contribution in this paper is a
deep fusion framework that more effectively exploits spatial features from CNNs
with temporal features from LSTM models. We also extensively evaluate their
strengths and weaknesses. We find that by combining both the sets of features,
the fully connected features effectively act as an attention mechanism to
direct the LSTM to interesting parts of the convolutional feature sequence. The
significance of our fusion method is its simplicity and effectiveness compared
to other state-of-the-art methods. The evaluation results demonstrate that this
hierarchical multi stream fusion method has higher performance compared to
single stream mapping methods allowing it to achieve high accuracy
outperforming current state-of-the-art methods in three widely used databases:
UCF11, UCFSports, jHMDB.

本文主要介绍了基于卷积神经网络和 LSTM 神经网络的动作识别方法，实验证明本文提出的多流融合模型可以有效地提高识别准确率。