We present the Verifee Dataset: a novel dataset of news articles with
fine-grained trustworthiness annotations. We develop a detailed methodology
that assesses the texts based on their parameters encompassing editorial
transparency, journalist conventions, and objective reporting while penalizing
manipulative techniques. We bring aboard a diverse set of researchers from
social, media, and computer sciences to overcome barriers and limited framing
of this interdisciplinary problem. We collect over $10,000$ unique articles
from almost $60$ Czech online news sources. These are categorized into one of
the $4$ classes across the credibility spectrum we propose, raging from
entirely trustworthy articles all the way to the manipulative ones. We produce
detailed statistics and study trends emerging throughout the set. Lastly, we
fine-tune multiple popular sequence-to-sequence language models using our
dataset on the trustworthiness classification task and report the best testing
F-1 score of $0.52$. We open-source the dataset, annotation methodology, and
annotators' instructions in full length at this https URL to
enable easy build-up work. We believe similar methods can help prevent
disinformation and educate in the realm of media literacy.

介绍 Verifee 数据集，其中分类了四种不同的新闻可信度等级，通过调整多种流行序列到序列语言模型，在信任度分类任务中达到了最佳的 F-1 得分 0.52，并通过开源数据集和注释方法有助于防止虚假报道和提高公众媒体素养。

细粒度捷克新闻文章数据集：信誉度分析的跨学科方法

Fine-grained Czech News Article Dataset: An Interdisciplinary Approach  to Trustworthiness Analysis

Recent events, such as the 2016 US Presidential Campaign, Brexit and the
COVID-19 "infodemic", have brought into the spotlight the dangers of online
disinformation. There has been a lot of research focusing on fact-checking and
disinformation detection. However, little attention has been paid to the
specific rhetorical and psychological techniques used to convey propaganda
messages. Revealing the use of such techniques can help promote media literacy
and critical thinking, and eventually contribute to limiting the impact of
"fake news" and disinformation campaigns. Prta (Propaganda Persuasion
Techniques Analyzer) allows users to explore the articles crawled on a regular
basis by highlighting the spans in which propaganda techniques occur and to
compare them on the basis of their use of propaganda techniques. The system
further reports statistics about the use of such techniques, overall and over
time, or according to filtering criteria specified by the user based on time
interval, keywords, and/or political orientation of the media. Moreover, it
allows users to analyze any text or URL through a dedicated interface or via an
API. The system is available online: this https URL

本研究关注于揭示传播宣传信息所使用的特殊修辞和心理技巧，以促进媒体素养和批判性思维，并最终有助于限制 “假新闻” 和虚假宣传活动的影响。Pra (Propaganda Persuasion Techniques Analyzer) 可探索定期抓取的文章，通过突出显示宣传技术出现的跨度进行比较，并报告相关统计数据。