Using vision-language models (VLMs) in web development presents a promising
strategy to increase efficiency and unblock no-code solutions: by providing a
screenshot or a sketch of a UI, a VLM could generate the code to reproduce it,
for instance in a language like HTML. Despite the advancements in VLMs for
various tasks, the specific challenge of converting a screenshot into a
corresponding HTML has been minimally explored. We posit that this is mainly
due to the absence of a suitable, high-quality dataset. This work introduces
WebSight, a synthetic dataset consisting of 2 million pairs of HTML codes and
their corresponding screenshots. We fine-tune a foundational VLM on our dataset
and show proficiency in converting webpage screenshots to functional HTML code.
To accelerate the research in this area, we open-source WebSight.

使用视觉 - 语言模型（VLMs）在 Web 开发中提供截屏或草图界面，能够自动生成相应的 HTML 代码，这项研究介绍了 WebSight 合成数据集和基于该数据集进行优化的 VLM，在将网页截屏转换为功能性 HTML 代码方面表现出良好的效果，并开源了 WebSight 以加速该领域的研究。