Building agents to interact with the web would allow for significant improvements in knowledge understanding and representation learning. However, web navigation tasks are difficult for current deep reinforcement learning (RL) models due to the large discrete action space and the varying number of actions between the states. In this work, we introduce DOM-Q-NET, a novel architecture for RL-based web navigation to address both of these problems. It parametrizes Q functions with separate networks for different action categories: clicking a DOM element and typing a string input. Our model utilizes a graph neural network to represent the tree-structured HTML of a standard web page. We demonstrate the capabilities of our model on the MiniWoB environment where we can match or outperform existing work without the use of expert demonstrations. Furthermore, we show 2x improvements in sample efficiency when training in the multi-task setting, allowing our model to transfer learned behaviours across tasks.

本文介绍DOM-Q-NET，一种基于深度强化学习和图神经网络的网页导航新架构。该架构通过将Q函数参数化为不同行动类别的网络，解决了当前深度强化学习模型在网页导航任务中因动作空间巨大和状态之间变化的不确定性等问题的难点。结果表明，该模型在MiniWoB环境中可以达到或者超过现有工作，同时在多任务设置中可以提高2倍的样本效率。

DOM-Q-NET:基于结构化语言的基础强化学习