Recent advancements in language models have demonstrated remarkable
improvements in various natural language processing (NLP) tasks such as web
navigation. Supervised learning (SL) approaches have achieved impressive
performance while utilizing significantly less training data compared to
previous methods. However, these SL-based models fall short when compared to
reinforcement learning (RL) approaches, which have shown superior results. In
this paper, we propose a novel approach that combines SL and RL techniques over
the MiniWoB benchmark to leverage the strengths of both methods. We also
address a critical limitation in previous models' understanding of HTML
content, revealing a tendency to memorize target elements rather than
comprehend the underlying structure. To rectify this, we propose methods to
enhance true understanding and present a new baseline of results. Our
experiments demonstrate that our approach outperforms previous SL methods on
certain tasks using less data and narrows the performance gap with RL models,
achieving 43.58\% average accuracy in SL and 36.69\% when combined with a
multimodal RL approach. This study sets a new direction for future web
navigation and offers insights into the limitations and potential of language
modeling for computer tasks.

该研究提出了一种将监督学习和强化学习技术相结合的新方法，通过在 MiniWoB 基准上利用两种方法的优势，解决了先前模型对 HTML 内容的理解上的关键限制，并展示了优于先前监督学习方法的实验结果，同时在与多模式强化学习方法相结合时缩小了与强化学习模型之间的性能差距，从而为未来的网络导航提供了新的方向和对语言建模在计算机任务中的潜力的洞察。