This paper examines the efficacy of utilizing large language models (LLMs) to
detect public threats posted online. Amid rising concerns over the spread of
threatening rhetoric and advance notices of violence, automated content
analysis techniques may aid in early identification and moderation. Custom data
collection tools were developed to amass post titles from a popular Korean
online community, comprising 500 non-threat examples and 20 threats. Various
LLMs (GPT-3.5, GPT-4, PaLM) were prompted to classify individual posts as
either "threat" or "safe." Statistical analysis found all models demonstrated
strong accuracy, passing chi-square goodness of fit tests for both threat and
non-threat identification. GPT-4 performed best overall with 97.9% non-threat
and 100% threat accuracy. Affordability analysis also showed PaLM API pricing
as highly cost-efficient. The findings indicate LLMs can effectively augment
human content moderation at scale to help mitigate emerging online risks.
However, biases, transparency, and ethical oversight remain vital
considerations before real-world implementation.

利用大型语言模型（LLMs）检测在线发布的公共威胁的有效性进行了研究。各种 LLMs 被用于分类帖子为 “威胁” 或 “安全”，结果表明 LLMs 能够有效地增强人类内容审查以帮助缓解新兴的在线风险。

利用大型语言模型检测在线发布的公共威胁的效力

Efficacy of Utilizing Large Language Models to Detect Public Threat  Posted Online

Diversity Searcher is a tool originally developed to help analyse diversity
in news media texts. It relies on a form of automated content analysis and thus
rests on prior assumptions and depends on certain design choices related to
diversity and fairness. One such design choice is the external knowledge
source(s) used. In this article, we discuss implications that these sources can
have on the results of content analysis. We compare two data sources that
Diversity Searcher has worked with - DBpedia and Wikidata - with respect to
their ontological coverage and diversity, and describe implications for the
resulting analyses of text corpora. We describe a case study of the relative
over- or under-representation of Belgian political parties between 1990 and
2020 in the English-language DBpedia, the Dutch-language DBpedia, and Wikidata,
and highlight the many decisions needed with regard to the design of this data
analysis and the assumptions behind it, as well as implications from the
results. In particular, we came across a staggering over-representation of the
political right in the English-language DBpedia.

本文比较了 Diversity Searcher 使用的两个数据源：DBpedia 和 Wikidata，讨论了这些数据源对文本分析结果的影响，描述了在设计数据分析和其背后的假设以及结果方面所需的许多决策和潜在影响。

DBpedia 和 Wikidata 中的政治代表性偏见对下游处理的挑战

Political representation bias in DBpedia and Wikidata as a challenge for downstream processing

The ever-increasing amount of user-generated content online has led, in
recent years, to an expansion in research and investment in automated content
analysis tools. Scrutiny of automated content analysis has accelerated during
the COVID-19 pandemic, as social networking services have placed a greater
reliance on these tools due to concerns about health risks to their moderation
staff from in-person work. At the same time, there are important policy debates
around the world about how to improve content moderation while protecting free
expression and privacy. In order to advance these debates, we need to
understand the potential role of automated content analysis tools.
This paper explains the capabilities and limitations of tools for analyzing
online multimedia content and highlights the potential risks of using these
tools at scale without accounting for their limitations. It focuses on two main
categories of tools: matching models and computer prediction models. Matching
models include cryptographic and perceptual hashing, which compare
user-generated content with existing and known content. Predictive models
(including computer vision and computer audition) are machine learning
techniques that aim to identify characteristics of new or previously unknown
content.

本文解释了在线多媒体内容分析工具的能力和局限性，并强调了在不考虑其局限性的情况下大规模使用这些工具的潜在风险，重点是两种主要工具：匹配模型和预测模型。

自动化多媒体内容分析的能力与局限性研究

Do You See What I See? Capabilities and Limits of Automated Multimedia Content Analysis

In recent years, fully automated content analysis based on probabilistic
topic models has become popular among social scientists because of their
scalability. The unsupervised nature of the models makes them suitable for
exploring topics in a corpus without prior knowledge. However, researchers find
that these models often fail to measure specific concepts of substantive
interest by inadvertently creating multiple topics with similar content and
combining distinct themes into a single topic. In this paper, we empirically
demonstrate that providing a small number of keywords can substantially enhance
the measurement performance of topic models. An important advantage of the
proposed keyword assisted topic model (keyATM) is that the specification of
keywords requires researchers to label topics prior to fitting a model to the
data. This contrasts with a widespread practice of post-hoc topic
interpretation and adjustments that compromises the objectivity of empirical
findings. In our application, we find that keyATM provides more interpretable
results, has better document classification performance, and is less sensitive
to the number of topics than the standard topic models. Finally, we show that
keyATM can also incorporate covariates and model time trends. An open-source
software package is available for implementing the proposed methodology.

本文提出了一种关键词辅助的主题模型，通过提供一小部分关键词，可以大大提高主题模型的测量性能，该模型具有更好的可解释性和文档分类性能，并且对主题数量的敏感性较小。