In this work a general framework is proposed to support the development of software systems that are able to adapt their behaviour according to the operating environment changes. The proposed approach, named REPTILE, works in a complete proactive manner and relies on Deep Reinforcement Learning-based agents to react to events, referred as novelties, that can affect the expected behaviour of the system. In our framework, two types of novelties are taken into account: those related to the context/environment and those related to the physical architecture itself. The framework, predicting those novelties before their occurrence, extracts time-changing models of the environment and uses a suitable Markov Decision Process to deal with the real-time setting. Moreover, the architecture of our RL agent evolves based on the possible actions that can be taken.

该论文提出了一种名为REPTILE的自适应软件系统的通用框架，它完全采用主动方式，并依赖于基于深度强化学习的代理程序来响应事件，称为新奇性，这些新奇性可能影响系统的预期行为。该框架考虑两种类型的新奇性：与环境相关的和与物理架构本身相关的，预测出这些新奇性，提取出环境的时间变化模型，并使用适当的马尔可夫决策过程来处理实时设置，此外，我们的RL代理的架构会根据可能采取的行动进化。

REPTILE: 一种主动实时深度强化学习自适应框架