Go-Explore achieved breakthrough performance on challenging reinforcement
learning (RL) tasks with sparse rewards. The key insight of Go-Explore was that
successful exploration requires an agent to first return to an interesting
state ('Go'), and only then explore into unknown terrain ('Explore'). We refer
to such exploration after a goal is reached as 'post-exploration'. In this
paper we present a systematic study of post-exploration, answering open
questions that the Go-Explore paper did not answer yet. First, we study the
isolated potential of post-exploration, by turning it on and off within the
same algorithm. Subsequently, we introduce new methodology to adaptively decide
when to post-explore and for how long to post-explore. Experiments on a range
of MiniGrid environments show that post-exploration indeed boosts performance
(with a bigger impact than tuning regular exploration parameters), and this
effect is further enhanced by adaptively deciding when and for how long to
post-explore. In short, our work identifies adaptive post-exploration as a
promising direction for RL exploration research.

本文对 Go-Explore 论文中未解决的问题进行了系统研究，通过对一系列 MiniGrid 环境的实验发现，采用后探索策略能够显著提高强化学习的性能，并利用新的自适应方法进一步提高了效果，证明了自适应后探索是强化学习探索研究的一个有前途的方向。

何时出发，何时探索：后探索对内在动机的益处

When to Go, and When to Explore: The Benefit of Post-Exploration in Intrinsic Motivation

Feature extraction from persistence diagrams, as a tool to enrich machine
learning techniques, has received increasing attention in recent years. In this
paper we explore an adaptive methodology to localize features in persistent
diagrams, which are then used in learning tasks. Specifically, we investigate
three algorithms, CDER, GMM and HDBSCAN, to obtain adaptive template
functions/features. Said features are evaluated in three classification
experiments with persistence diagrams. Namely, manifold, human shapes and
protein classification. The main conclusion of our analysis is that adaptive
template systems, as a feature extraction technique, yield competitive and
often superior results in the studied examples. Moreover, from the adaptive
algorithms here studied, CDER consistently provides the most reliable and
robust adaptive featurization.

探索一种自适应方法来在持久图中定位特征，并评估 3 种算法在分类实验中的性能，结果表明自适应模板系统可以作为一种竞争性和优越的特征提取技术，其中 CDER 算法一致表现最可靠和稳健。

自适应模板系统：基于数据驱动的持久图学习特征选择

Adaptive template systems: Data-driven feature selection for learning  with persistence diagrams

We consider penalized estimation in hidden Markov models (HMMs) with
multivariate Normal observations. In the moderate-to-large dimensional setting,
estimation for HMMs remains challenging in practice, due to several concerns
arising from the hidden nature of the states. We address these concerns by
$\ell_1$-penalization of state-specific inverse covariance matrices. Penalized
estimation leads to sparse inverse covariance matrices which can be interpreted
as state-specific conditional independence graphs. Penalization is nontrivial
in this latent variable setting; we propose a penalty that automatically adapts
to the number of states $K$ and the state-specific sample sizes and can cope
with scaling issues arising from the unknown states. The methodology is
adaptive and very general, applying in particular to both low- and
high-dimensional settings without requiring hand tuning. Furthermore, our
approach facilitates exploration of the number of states $K$ by coupling
estimation for successive candidate values $K$. Empirical results on simulated
examples demonstrate the effectiveness of the proposed approach. In a
challenging real data example from genome biology, we demonstrate the ability
of our approach to yield gains in predictive power and to deliver richer
estimates than existing methods.

本文提出了一种基于拉格朗日惩罚项的自适应笛卡尔积方法，该方法可以应用于多元正态观测下的隐马尔可夫模型参数估计，以获得稀疏的逆协方差矩阵，并能在不需要手动调整超参数的情况下适用于低维和高维数据，并能有效处理实际应用中的数据预测与生物遗传学领域等问题。