The proliferation of the Internet of Things (IoT) has led to an explosion of
data generated by interconnected devices, presenting both opportunities and
challenges for intelligent decision-making in complex environments. Traditional
Reinforcement Learning (RL) approaches often struggle to fully harness this
data due to their limited ability to process and interpret the intricate
patterns and dependencies inherent in IoT applications. This paper introduces a
novel framework that integrates transformer architectures with Proximal Policy
Optimization (PPO) to address these challenges. By leveraging the
self-attention mechanism of transformers, our approach enhances RL agents'
capacity for understanding and acting within dynamic IoT environments, leading
to improved decision-making processes. We demonstrate the effectiveness of our
method across various IoT scenarios, from smart home automation to industrial
control systems, showing marked improvements in decision-making efficiency and
adaptability. Our contributions include a detailed exploration of the
transformer's role in processing heterogeneous IoT data, a comprehensive
evaluation of the framework's performance in diverse environments, and a
benchmark against traditional RL methods. The results indicate significant
advancements in enabling RL agents to navigate the complexities of IoT
ecosystems, highlighting the potential of our approach to revolutionize
intelligent automation and decision-making in the IoT landscape.

通过将 transformer 架构与 Proximal Policy Optimization（PPO）相结合，本文引入了一种新颖的框架来解决物联网应用中复杂环境下智能决策中的挑战，通过利用 transformer 的自注意机制，增强了强化学习代理在动态物联网环境中的理解和行动能力，从而提高了决策过程的效率。通过在智能家居自动化和工业控制系统等各种物联网场景中进行实证实验证明了该方法的有效性，显示了决策效率和适应性的显著提高。该研究的贡献包括对处理异构物联网数据中 transformer 的作用进行详尽的探索，对框架在不同环境中性能的全面评估，以及与传统强化学习方法进行的基准测试。结果表明该方法显著提高了强化学习代理在物联网生态系统中应对复杂性的能力，凸显了该方法在物联网领域智能自动化和决策制定领域的潜力。