Traditional search systems focus on query formulation for effective results
but face challenges in scenarios such as product searches where crucial product
details (e.g., size, color) remain concealed until users visit specific product
pages. This highlights the need for intelligent web navigation agents capable
of formulating queries and navigating web pages according to users' high-level
intents. In response to this need, this work introduces a Grounded Language
Agent for Intelligent Web Interactions, called GLAINTEL. Drawing upon
advancements in language modeling and reinforcement learning, GLAINTEL
investigates the efficacy of transformer-based models in enhancing the search
capabilities of interactive web environments. Given the dynamic action space
for each state in web navigation, GLAINTEL employs the Flan-T5 architecture and
incorporates language modeling and value estimation heads. This work focuses on
training smaller language models as agents across various scenarios,
systematically evaluating the impact of human demonstrations on the training
process. Specifically, we investigate scenarios where no human demonstrations
are available and subsequently assess the effective utilization of such
demonstrations. We also explore unsupervised domain adaptation for situations
where demonstrations are confined to a specific domain. Experimental
evaluations across diverse setups demonstrate the effectiveness of training
agents in unsupervised settings, outperforming in-context learning-based
approaches that employ larger models with up to 540 billion parameters.
Surprisingly, behavioral cloning-based methods that straightforwardly use human
demonstrations do not outperform unsupervised learning-based methods.
Additionally, combining human demonstrations with Reinforcement Learning-based
training yields results comparable to models utilizing GPT-4.

针对传统搜索系统在产品搜索等场景中面临的挑战，本研究介绍了一种用于智能网页交互的基于语言模型和强化学习的框架，名为 GLAINTEL。GLAINTEL 通过引入基于 transformer 模型的 Flan-T5 架构，结合语言建模和价值评估模块，实现了对网页环境中搜索能力的增强。通过系统评估不同场景下的训练效果，发现在无人示范的情况下，无监督学习方法的效果优于行为克隆方法，同时将人类示范与强化学习相结合的模型效果与使用 GPT-4 模型的结果相当。