In this position paper, we discuss the potential for leveraging LLMs as
interactive research tools to facilitate collaboration between human coders and
AI to effectively annotate online risk data at scale. Collaborative human-AI
labeling is a promising approach to annotating large-scale and complex data for
various tasks. Yet, tools and methods to support effective human-AI
collaboration for data annotation are under-studied. This gap is pertinent
because co-labeling tasks need to support a two-way interactive discussion that
can add nuance and context, particularly in the context of online risk, which
is highly subjective and contextualized. Therefore, we provide some of the
early benefits and challenges of using LLMs-based tools for risk annotation and
suggest future directions for the HCI research community to leverage LLMs as
research tools to facilitate human-AI collaboration in contextualized online
data annotation. Our research interests align very well with the purposes of
the LLMs as Research Tools workshop to identify ongoing applications and
challenges of using LLMs to work with data in HCI research. We anticipate
learning valuable insights from organizers and participants into how LLMs can
help reshape the HCI community's methods for working with data.

利用 LLMs 作为交互式研究工具，促进人类编码员与人工智能之间的合作，有效地标记大规模在线风险数据。