Offline meta reinforcement learning (OMRL) has emerged as a promising
approach for interaction avoidance and strong generalization performance by
leveraging pre-collected data and meta-learning techniques. Previous
context-based approaches predominantly rely on the intuition that maximizing
the mutual information between the task and the task representation ($I(Z;M)$)
can lead to performance improvements. Despite achieving attractive results, the
theoretical justification of performance improvement for such intuition has
been lacking. Motivated by the return discrepancy scheme in the model-based RL
field, we find that maximizing $I(Z;M)$ can be interpreted as consistently
raising the lower bound of the expected return for a given policy conditioning
on the optimal task representation. However, this optimization process ignores
the task representation shift between two consecutive updates, which may lead
to performance improvement collapse. To address this problem, we turn to use
the framework of performance difference bound to consider the impacts of task
representation shift explicitly. We demonstrate that by reining the task
representation shift, it is possible to achieve monotonic performance
improvements, thereby showcasing the advantage against previous approaches. To
make it practical, we design an easy yet highly effective algorithm RETRO
(\underline{RE}ining \underline{T}ask \underline{R}epresentation shift in
context-based \underline{O}ffline meta reinforcement learning) with only adding
one line of code compared to the backbone. Empirical results validate its
state-of-the-art (SOTA) asymptotic performance, training stability and
training-time consumption on MuJoCo and MetaWorld benchmarks.

通过最大化互信息来提高任务表示能够实现性能的单调改善，其中，RETRO 算法重新调整任务表示偏移，从而在离线元强化学习中取得了 SOTA 的渐近性能、训练稳定性和训练时间消耗的实证结果。