Automated classifiers (ACs), often built via supervised machine learning
(SML), can categorize large, statistically powerful samples of data ranging
from text to images and video, and have become widely popular measurement
devices in communication science and related fields. Despite this popularity,
even highly accurate classifiers make errors that cause misclassification bias
and misleading results in downstream analyses-unless such analyses account for
these errors. As we show in a systematic literature review of SML applications,
communication scholars largely ignore misclassification bias. In principle,
existing statistical methods can use "gold standard" validation data, such as
that created by human annotators, to correct misclassification bias and produce
consistent estimates. We introduce and test such methods, including a new
method we design and implement in the R package misclassificationmodels, via
Monte Carlo simulations designed to reveal each method's limitations, which we
also release. Based on our results, we recommend our new error correction
method as it is versatile and efficient. In sum, automated classifiers, even
those below common accuracy standards or making systematic misclassifications,
can be useful for measurement with careful study design and appropriate error
correction methods.

通过系统文献综述，我们发现大多数通信学者没有考虑分类器的误分类偏差。我们介绍并测试了 “gold standard” 验证数据的错误校正方法，并通过蒙特卡罗模拟来揭示每种方法的局限性。我们推荐我们设计和实施的新方法作为更有效的误差校正方法。