Large Language Models (LLMs) are promising analytical tools. They can augment
human epistemic, cognitive and reasoning abilities, and support 'sensemaking',
making sense of a complex environment or subject by analysing large volumes of
data with a sensitivity to context and nuance absent in earlier text processing
systems. This paper presents a pilot experiment that explores how LLMs can
support thematic analysis of controversial topics. We compare how human
researchers and two LLMs GPT-4 and Llama 2 categorise excerpts from media
coverage of the controversial Australian Robodebt scandal. Our findings
highlight intriguing overlaps and variances in thematic categorisation between
human and machine agents, and suggest where LLMs can be effective in supporting
forms of discourse and thematic analysis. We argue LLMs should be used to
augment, and not replace human interpretation, and we add further
methodological insights and reflections to existing research on the application
of automation to qualitative research methods. We also introduce a novel
card-based design toolkit, for both researchers and practitioners to further
interrogate LLMs as analytical tools.

该研究试验了大型语言模型（LLMs）在支持主题分析上的作用，发现人工研究者与两个 LLMs 在对澳大利亚 Robodebt 丑闻媒体报道的片段进行主题分类时存在有趣的重叠和差异，表明 LLMs 在支持对话和主题分析方面可以发挥作用，但应用中应该作为人类解释的补充，同时该研究还介绍了一种新颖的基于卡片的设计工具箱，以进一步研究 LLMs 作为分析工具的潜力。

主题分析的自动化：LLM 如何分析争议话题

Automating Thematic Analysis: How LLMs Analyse Controversial Topics

Is tractable tokenization for humans also tractable for machine learning
models? This study investigates relations between tractable tokenization for
humans (e.g., appropriateness and readability) and one for models of machine
learning (e.g., performance on an NLP task). We compared six tokenization
methods on the Japanese commonsense question-answering dataset (JCommmonsenseQA
in JGLUE). We tokenized question texts of the QA dataset with different
tokenizers and compared the performance of human annotators and
machine-learning models. Besides,we analyze relationships among the
performance, appropriateness of tokenization, and response time to questions.
This paper provides a quantitative investigation result that shows the
tractable tokenizations for humans and machine learning models are not
necessarily the same as each other.

本研究比较了六种不同的 tokenization 方法在日语常识问答数据集上人工注释和机器学习模型的表现，分析了 tokenization 的合适性和响应时间等因素和表现的关系，结果表明人类和机器学习模型的 tokenization 不完全一致。