Knowledge workers frequently encounter repetitive web data entry tasks, like
updating records or placing orders. Web automation increases productivity, but
translating tasks to web actions accurately and extending to new specifications
is challenging. Existing tools can automate tasks that perform the same logical
trace of UI actions (e.g., input text in each field in order), but do not
support tasks requiring different executions based on varied input conditions.
We present DiLogics, a programming-by-demonstration system that utilizes NLP to
assist users in creating web automation programs that handle diverse
specifications. DiLogics first semantically segments input data to structured
task steps. By recording user demonstrations for each step, DiLogics
generalizes the web macros to novel but semantically similar task requirements.
Our evaluation showed that non-experts can effectively use DiLogics to create
automation programs that fulfill diverse input instructions. DiLogics provides
an efficient, intuitive, and expressive method for developing web automation
programs satisfying diverse specifications.

DiLogics 是一种可以利用自然语言处理（NLP）辅助用户创建处理各种规范的 Web 自动化程序的演示编程系统，它通过将输入数据语义分段为结构化任务步骤，并记录用户演示来泛化 Web 宏以满足新的但语义相似的任务需求。我们的评估表明，非专业人士可以有效地使用 DiLogics 创建满足多样输入指令的自动化程序。DiLogics 提供了一种高效、直观和富有表现力的方法来开发满足不同规范的 Web 自动化程序。

DiLogics: 利用多种逻辑创建网络自动化程序

DiLogics: Creating Web Automation Programs With Diverse Logics

Natural language programming is a promising approach to enable end users to
instruct new tasks for intelligent agents. However, our formative study found
that end users would often use unclear, ambiguous or vague concepts when
naturally instructing tasks in natural language, especially when specifying
conditionals. Existing systems have limited support for letting the user teach
agents new concepts or explaining unclear concepts. In this paper, we describe
a new multi-modal domain-independent approach that combines natural language
programming and programming-by-demonstration to allow users to first naturally
describe tasks and associated conditions at a high level, and then collaborate
with the agent to recursively resolve any ambiguities or vagueness through
conversations and demonstrations. Users can also define new procedures and
concepts by demonstrating and referring to contents within GUIs of existing
mobile apps. We demonstrate this approach in PUMICE, an end-user programmable
agent that implements this approach. A lab study with 10 users showed its
usability.

本篇论文介绍了一种新的多模态、领域无关的方法，结合自然语言编程和演示编程，允许用户在高层次上自然描述任务和相关条件，并通过对话和演示递归地解决任何模糊不清的问题。PUMICE 是一个可供最终用户编程的代理系统，实现了这种方法。10 位用户的实验显示出系统的可用性。