Large language and vision models have transformed how social movements
scholars identify protest and extract key protest attributes from multi-modal
data such as texts, images, and videos. This article documents how we
fine-tuned two large pretrained transformer models, including longformer and
swin-transformer v2, to infer potential protests in news articles using textual
and imagery data. First, the longformer model was fine-tuned using the Dynamic
of Collective Action (DoCA) Corpus. We matched the New York Times articles with
the DoCA database to obtain a training dataset for downstream tasks. Second,
the swin-transformer v2 models was trained on UCLA-protest imagery data.
UCLA-protest project contains labeled imagery data with information such as
protest, violence, and sign. Both fine-tuned models will be available via
https://github.com/Joshzyj/llvms4protest. We release this short technical
report for social movement scholars who are interested in using LLVMs to infer
protests in textual and imagery data.

大型语言和视觉模型已经改变了社会运动学者如何识别抗议活动并从多模态数据中提取关键的抗议属性。本文描述了我们如何通过对大规模预训练的转换器模型（包括 longformer 和 swin-transformer v2）进行微调，使用文本和图像数据来推断新闻文章中的潜在抗议活动。我们为下游任务使用 Dynamic of Collective Action (DoCA) Corpus 训练了 longformer 模型，并将纽约时报文章与 DoCA 数据库匹配，以获取训练数据集。我们还使用了 UCLA-protest 图像数据对 swin-transformer v2 模型进行了训练。我们通过 https://github.com/Joshzyj/llvms4protest 发布了这篇简短的技术报告，供对使用 LLVMs 来推断文本和图像数据中的抗议活动感兴趣的社会运动学者使用。